DevOps for Java Developers

DevOps and DevSecOps best practices for

Java developers

Neeraj Kumar

[Link]
First Edition 2025

Copyright © BPB Publications, India

ISBN: 978-93-65896-015

All Rights Reserved. No part of this publication may be reproduced,

distributed or transmitted in any form or by any means or stored in a
database or retrieval system, without the prior written permission of the
publisher with the exception to the program listings which may be entered,
stored and executed in a computer system, but they can not be reproduced
by the means of publication, photocopy, recording, or by any electronic and
mechanical means.

LIMITS OF LIABILITY AND DISCLAIMER OF WARRANTY

The information contained in this book is true to correct and the best of
author’s and publisher’s knowledge. The author has made every effort to
ensure the accuracy of these publications, but publisher cannot be held
responsible for any loss or damage arising from any information in this
book.

All trademarks referred to in the book are acknowledged as properties of

their respective owners but BPB Publications cannot guarantee the accuracy
of this information.

[Link]
Dedicated to
My father
Dr. Nagendra Kumar
and
My mother
Daisy Gupta
 About the Author

Neeraj Kumar has been working in software industry for more than 16
years, playing central roles in numerous projects as a technical leader,
DevOps engineer and software engineer, delivering projects using
automation engineering for big companies, including successful projects in
Australia, South America, United Kingdom, Europe, and the United States.
Currently, he is a senior DevOps architect/engineer and a technical advisor
at one of the leading product based company. He is also an accomplished
postgraduate completing a degree in computer applications and systems and
holds two master’s degrees focused on software engineering (M.C.A) with
engineering practices (M.B.A). In the meantime, he successfully got many
certifications in cloud technologies and DevOps. Furthermore, the author
participates as a speaker in various IT conferences and writes technical
articles on DevOps and related topics. He also provides DevOps trainings to
teams and individuals.
 About the Reviewer

Ron is a seasoned software engineer with over 20 years of experience in the

Java ecosystem. From mainframes to Microservices, he's seen it all. His
passion for software engineering and architecture drives his work.
A certified Java expert (OCP and SCBCD/OCPBCD), Ron is proficient in a
wide range of frameworks and libraries, from Apache to ZK. He's also keen
on exploring alternative JVM languages like Kotlin.
As a special agent and senior developer at Team Rockstars IT, Ron shares
his knowledge as an international conference speaker and via his written
works.
He's the author of books 'Java Cloud-native migrations with Jakarta EE' and
'Virtual Threads, Structured Concurrency, and Scoped Values'.
 Acknowledgement

I want to express my deepest gratitude to my parents, whose guidance and

support have been a cornerstone in my life, and to my family and friends for
their unwavering support and encouragement throughout this book's
writing. A special thanks goes to my incredible wife Kumud, whose love,
patience and belief in me have been my true anchor on this journey. Also, I
am deeply thankful for the constant support of my amazing sisters Anju and
Anu.
I am also grateful to BPB Publications for their guidance and expertise in
bringing this book to fruition. It was a long journey of revising this book,
with valuable participation and collaboration of reviewers, technical
experts, and editors.
I would also like to acknowledge the valuable contributions of my
colleagues and co-worker during many years working in the tech industry,
who have taught me so much and provided valuable feedback on my work.
Finally, I would like to thank all the readers who have taken an interest in
my book and for their support in making it a reality. Your encouragement
has been invaluable.
 Preface

With rise of DevOps, cloud platform, and container technologies, the way
we build, deploy, and secure applications is rapidly changing. DevOps
brings together the essential practices, philosophies, and tools that
streamline software creation and making it possible to develop and release
the features with pace.
This book aims to help you in understanding and using the latest DevOps
techniques in simplifying your build and deployment process using
Microservices, serverless, and cloud native technologies. It covers a wide
range of topics, including the principles of DevOps, source code
management, CI/CD, containerization, package management and
DevSecOps, gaining insights into the essential tools that needed to build
and deploy modern applications. For anyone developing with Spring boot,
Micronaut, Quarkaus, and more, this book provides a solid foundation.
As mobile devices are increasing, Mobile DevOps has become essential for
the teams focused on creating a seamless user experience across platforms.
This book dedicates a full chapter to Mobile DevOps, exploring its best
practices and strategies for integrating DevOps into mobile development
workflows.
Throughout the book, you will learn about the key principles, best practices
and various tools to implement DevOps in your software development
tasks. Each chapter includes hands-on insights and real-world examples,
guiding you through the DevOps lifecycle-from building applications to
securing and managing the artifacts.
Whether you’re a software developer looking to streamline your workflow
or a DevOps professional aiming to build resilient, efficient applications,
this book offers the tools and insight you need to succeed. It’s my hope that
this book will empower you to embrace DevOps, improve your process, and
make your mark in modern software development.
Chapter 1: Lead the Transformation with DevOps – provides a thorough
understanding of DevOps by exploring its key concepts, importance,
benefits and practical applications. The chapter also explains the role of
automation in DevOps, the structure of DevOps team and DevOps strategy.
Furthermore, the chapter also reviews the DevOps process, focus
particularly on continuous integration and continuous delivery (CI/CD).
Chapter 2: Slicing the Monolith - highlights the needs for flexibility,
scalability and agility in software development and presents a detailed
overview of shifting from Monolithic architecture to Microservices. The
chapter outlines the key differences between the two, addressing the
challenges of Monolithic systems and when Microservices are preferable.
You'll find practical guidance for a smooth transition, best practices, and
insights on how DevOps supports this shift. The chapter also covers
frameworks like Spring Boot, Micronaut, and Quarkus, along with
serverless architecture using AWS Lambda.
Chapter 3: Manage Your Source Code - covers the basics of source
control management, its historical evolution, and the shift towards
distributed version control, particularly focusing on Git. You'll learn about
code versioning best practices, how to choose the right source control tool,
and the components of Git, including creating your first Git pull request.
We explore various Git clients and workflows, wrapping up with essential
best practices for effective source control management.
Chapter 4: Containerization – explores the transformative role of
containers in application development and management, emphasizing their
lightweight and consistent nature across various environments. The chapter
explains key concept of containers , it’s evolution and how it’s distinguish
from virtual machines. Furthermore, chapter also covers Docker
architecture and Kubernetes orchestration, with practical guidance on
setting up Docker and executing basic commands. The chapter also
addresses essential topics like tagging, versioning, and image layers and
best practices for building optimized container images.
Chapter 5: Continuous Integration – highlights continuous integration
(CI) as a foundational practice in software development, emphasizing its
importance in enhancing efficiency and early error detection. The chapter
covers key CI concepts and practices, focusing on declarative build scripts
with tools like Ant, Maven, and Gradle for streamlined build processes. The
chapter also covers continuous build processes and their role in supporting
agile development, along with the essential role of test automation to ensure
reliability. Furthermore, this chapter also discusses how to build a robust CI
pipeline using Jenkins and strategies to implement and maintain effective
CI workflows.
Chapter 6: Package Management - emphasizes that software artifacts
alone aren’t enough to ensure quality or competitive advantage—metadata
is essential for tracking build details, dependency versions, and
environments, enhancing the reproducibility and stability of build pipelines.
The chapter covers dependency resolution basics using Java tools like
Maven and Gradle, as well as managing dependencies for containers. The
chapter also provides guidance on publishing Java artifacts to repositories,
discussing Maven Central as a primary option and alternatives like
Sonatype Nexus and JFrog Artifactory for internal artifact management.
The chapter serves as a comprehensive guide to artifact generation,
metadata, dependency management, and publication best practices.
Chapter 7: Protecting Your Binaries - explains the importance of
embedding security early in the software development process to address
rising supply chain threats. The chapter covers key DevSecOps principles,
emphasizing a "shift-left" approach to proactively reduce vulnerabilities.
The chapter also allows readers to learn essential security methodologies
like SAST and DAST, as well as tools like SonarQube for ongoing security
checks to enhance security, minimize risks, and build resilient software
adaptable to evolving cyber threats.
Chapter 8: Deployment Strategies - provides the readers with essential
skills for deploying applications, particularly in Kubernetes environments.
The chapter covers planning deployment strategies, building and pushing
container images with Java tools like Jib and Eclipse JKube, and generating
Kubernetes manifests using Dekorate. The chapter also discusses key
deployment strategies, including rolling updates, blue-green, and canary
deployments. The chapter also emphasizes health checks, system
monitoring, and resource adjustment to maintain performance. Furthermore,
it explores high availability and multi-cloud strategies to handle complex
deployment scenarios.
Chapter 9: Continuous Delivery and Deployment – explains the
importance of continuous uptime, continuous integration (CI), continuous
delivery (CD), and continuous deployment in software development. You
will get to know the difference between continuous delivery and continuous
deployment. The chapter also covers key differences, best practices, and
challenges for continuous delivery and continuous deployment, with a
Jenkins pipeline example to demonstrate CI/CD in action.
Chapter 10: Mobile DevOps - is dedicated to the Mobile DevOps,
focusing on the distinct challenges of Android and iOS. The chapter
explains Android’s fragmented ecosystem, with thousands of devices from
multiple manufacturers and demand of a highly automated DevOps pipeline
for android and iOS. The chapter also covers robust CI/CD pipelines,
continuous testing, and UI automation on real devices to ensure reliable
user experiences. Furthermore the chapter highlights best Mobile DevOps
practices.
 Code Bundle and Coloured Images
Please follow the link to download the
Code Bundle and the Coloured Images of the book:

[Link]
The code bundle for the book is also hosted on GitHub at
[Link]
In case there’s an update to the code, it will be updated on the existing
GitHub repository.
We have code bundles from our rich catalogue of books and videos
available at [Link] Check them out!

Errata
We take immense pride in our work at BPB Publications and follow best
practices to ensure the accuracy of our content to provide with an indulging
reading experience to our subscribers. Our readers are our mirrors, and we
use their inputs to reflect and improve upon human errors, if any, that may
have occurred during the publishing processes involved. To let us maintain
the quality and help us reach out to any readers who might be having
difficulties due to any unforeseen errors, please write to us at :
errata@[Link]
Your support, suggestions and feedbacks are highly appreciated by the BPB
Publications’ Family.

Did you know that BPB offers eBook versions of every book
published, with PDF and ePub files available? You can upgrade to
the eBook version at [Link] and as a print book
 customer, you are entitled to a discount on the eBook copy. Get in
touch with us at :
business@[Link] for more details.
At [Link], you can also read a collection of free
technical articles, sign up for a range of free newsletters, and
receive exclusive discounts and offers on BPB books and eBooks.

Piracy
If you come across any illegal copies of our works in any form on the
internet, we would be grateful if you would provide us with the
location address or website name. Please contact us at
business@[Link] with a link to the material.

If you are interested in becoming an author

If there is a topic that you have expertise in, and you are interested in
either writing or contributing to a book, please visit
[Link]. We have worked with thousands of developers
and tech professionals, just like you, to help them share their insights
with the global tech community. You can make a general application,
apply for a specific hot topic that we are recruiting an author for, or
submit your own idea.

Reviews
Please leave a review. Once you have read and used this book, why
not leave a review on the site that you purchased it from? Potential
readers can then see and use your unbiased opinion to make purchase
decisions. We at BPB can understand what you think about our
products, and our authors can see your feedback on their book. Thank
you!
 For more information about BPB, please visit [Link].

Join our book’s Discord space

Join the book’s Discord Workspace for Latest updates, Offers, Tech
happenings around the world, New Release and Sessions with the Authors:
[Link]
 Table of Contents

1. Lead the Transformation with DevOps

Introduction
Structure
Objectives
Understanding DevOps
The need for DevOps
Business need
Application development need
Scenario 1
Scenario 2
DevOps and automation
DevOps team structure
Team roles
DevOps strategy
Benefits of DevOps
DevOps process
Source code management
Source code review
Configuration management
Build management
Artifacts repository management
Release management
 Test automation
Continuous integration
Continuous delivery
Continuous deployment
Infrastructure as code
Application performance monitoring
Conclusion

2. Slicing the Monolith

Introduction
Structure
Objectives
Monolithic vs. Microservices based architecture
Monolithic architecture
Microservices architecture
The need for Microservices
Planning Monolithic to Microservices migration
DevOps and Microservices
Understanding DevOps and Microservices
DevOps is the cultural shift
Microservices is the modular approach
The synergy of DevOps and Microservices
Microservices frameworks
Spring Boot
Micronaut
Quarkus
Serverless
Setting up
 Conclusion

3. Manage Your Source Code

Introduction
Structure
Objectives
Source control management
Three generations of source control management
First generation: Local SCM software
Second generation: Centralized SCM tools
Third generation: Distributed SCM systems
Code versioning
Semantic versioning
Best practices for code versioning
Choosing your source control
Git components
Git server
Git clients
Making your first pull request
Git tools
Git command-line basics
Preparing to work with Git from command line
Working with a local repository
Creating an initial repository
Git command-line tutorial
Git clients
Git IDE integration
Git workflows
 Centralized workflow
Feature branching
Git-flow workflow
Forking workflow
Trunk-based development
Best practices for source control management
Conclusion

4. Containerization
Introduction
Structure
Objectives
Understanding containers and their usage
History of containers
Difference between containers and virtual machines
Understanding virtual machines
Working of virtual machine
Components of virtual machine
Pros and cons of virtual machine
Understanding container
Working of containers
Pros and cons of container
Choosing VMs vs containers
Key container and image terminologies
Docker architecture
Dockerfile
Sample Dockerfile
Docker container orchestration using Kubernetes
 Working of Kubernetes
Docker on your machine
Basic tagging and image version management
Adding tags to images
Image layers
Run Kubernetes locally
Best image building practices
Conclusion

5. Continuous Integration
Introduction
Structure
Objectives
Continuous integration
Continuous integration in development team
Critical concepts of continuous integration
Key practices of continuous integration
Declarative build script
Build with Apache Ant
Key Ant terminology
Build with Apache Maven
Key Maven terminology
Build with Gradle
Key gradle terminology
Continuous build
Test automation
Monitor and maintain tests
Building pipeline with Jenkins
 Conclusion

6. Package Management
Introduction
Structure
Objectives
Understanding metadata
Key attributes of insightful metadata
Creating metadata
Capturing metadata
Writing the metadata
Understanding artifact
Package management
Dependency management
Dependency management with Apache Maven
Dependency management with Gradle
Dependency management in containers
Basics of dependency management in containers
Best practices for dependency management in containers
Artifact publication
Artifact publication and its benefits
Best practices of artifact publication
Publishing to Maven Local
Publishing to Maven Central
Publishing to Sonatype Nexus repository
Publishing to JFrog Artifactory
Conclusion

7. Protecting Your Binaries

Introduction
Structure
Objectives
Supply chain security compromised
Supply chain security from vendor perspective
Supply chain security from customer perspective
Full impact graph of supply chain security
Software security and why we need it
Understanding software security
Need for software security
Key elements of software security
Challenges in software security
DevSecOps
Components of DevSecOps
Benefits of DevSecOps
Challenges of adopting DevSecOps
Shift-left Security approach
Understanding the Shift-left Security approach
Key principles of the Shift-left Security approach
Benefits of the Shift-left Security approach
Challenges and considerations
Static and dynamic security analysis
Static application security testing
Understanding how SAST works
Benefits of SAST
Disadvantages of the SAST approach
Dynamic application security testing
Understanding how DAST work
 Benefits of using DAST
Disadvantages of DAST
Comparing SAST and DAST
When to use SAST and DAST
When to use SAST
When to use DAST
Can SAST and DAST work together
Interactive application security testing
Runtime application self-protection
Planning for DevSecOps pipeline
Vulnerability scoring system
Common vulnerability scoring system
Components of CVSS
CVSS in practice
Quality gate methodology
Importance of quality gates
Quality gate strategies
Implementing quality gates
Practical applications of quality management
Quality management with SonarQube
Jenkins pipeline to perform SonarQube analysis
Implementing security with the quality gate method
Risk management in quality gates
Conclusion

8. Deployment Strategies
Introduction
Structure
Objectives
Planning the deployment
Traditional deployment
Kubernetes deployment
Planning the deployment
Building and pushing container images
Managing container images by using JIB
Setting up Jib in your Maven project
Setting up Jib in your Gradle project
Building container images with Eclipse JKube
Kubernetes deployments
Local setup for deployment
Generate Kubernetes manifests using Dekorate
Generate and deploy Kubernetes manifests with Eclipse JKube
Choose and implement a deployment strategy
Managing workloads in Kubernetes
Setting up health checks
Adjusting resource quotas
Application configuration level
Kubernetes deployment configuration level
Working with persistent data collections
Best practices for logging and monitoring
Monitoring
Logging
Tracing
High availability and geographically distributed system
Hybrid and multicloud architectures
Hybrid architecture
 Multicloud architecture
Conclusion

9. Continuous Delivery and Deployment

Introduction
Structure
Objectives
Continuous delivery:
Continuous integration recap
The evolution to continuous delivery
Key components of continuous delivery
Benefits of continuous delivery
Need for continuous updates
Meeting evolving user expectations
Continuous uptime
Understanding continuous uptime
Necessity of continuous uptime
Continuous delivery and continuous deployments
Continuous delivery: Streamlining the delivery pipeline
Importance of continuous delivery
Best practices for continuous delivery
Challenges of continuous delivery
Continuous deployment: Automating the release pipeline
Importance of continuous deployment
Best practices for continuous delivery
Challenges of continuous delivery
Continuous delivery vs. continuous deployment
Choosing the right approach
 CI/CD pipeline example
Conclusion

10. Mobile DevOps

Introduction
Structure
Objectives
Introduction to Mobile DevOps
DevOps vs. Mobile DevOps
Necessity of Mobile DevOps
DevOps workflow for mobile
Mobile device fragmentation
iOS vs. Android fragmentation
OS fragmentation for iOS
The orchestrated iOS ecosystem
iOS version adoption
Legacy devices and tailored experiences
Best practices for iOS fragmentation
Navigating the complexity of OS fragmentation for Android
Android for disparate screens
Advanced hardware and 3D support
Challenges of implementing DevOps to mobile
Mobile CI/CD pipeline in the cloud
Real-world example
Mobile DevOps best practices
Continuous testing on parallel devices
Benefits of continuous testing on parallel devices
Conclusion
Index
 CHAPTER 1
Lead the Transformation with
DevOps

Introduction
In the traditional application development approach, multiple stakeholders
and departments, including different vendors, are involved in the entire
software development life cycle (SDLC). Different stages of application
development life cycle management are as follows:
• Business analysts gather the business requirements.
• The development team does the application development (this can
be outsourced).
• QA team does the testing (also can be outsourced) to check the
functional correctness and fitness for the purpose.
• Performance and stress testing are also performed in required
scenarios by suitable groups using appropriate tools.
• Then, the production deployment process, with proper checklist and
approvals managed by the IT operations teams.
• Maintenance teams do monitoring and support.
Structure
You can see that each stage of the maturity life cycle, from functionality
development to maintenance, is managed in silos by independent teams,
departments, processes, and tools. This approach is often fragmented by
frameworks, techniques, processes, people, and various tools. It impacts the
final product in terms of features, quality, cost, schedule, performance, and
other organizational overheads, such as interfacing and integration between
vendors. The maintenance, support costs, and skill needs in this method are
often overlooked. However, from the application development life cycle
and business points of view, maintenance and support activities are
important to evaluate the current system, and if any changes are required,
then we can estimate it well in advance.
In this chapter, we will discuss the following topics:
• Understanding DevOps
• The need for DevOps
• DevOps and automation
• DevOps team structure
• DevOps strategy
• Benefits of DevOps
• DevOps process

Objectives
By the end of this chapter, you will gain a thorough understanding of
DevOps by exploring its key concepts, importance, and practical
applications. We will start by defining DevOps and discussing why it is
essential in modern software development. Additionally, we will examine
the role of automation in DevOps, highlighting its impact on streamlining
workflows and improving efficiency.
This chapter will also cover the structure of DevOps team, highlighting the
importance of collaboration and cross-functional roles in driving successful
DevOps practices. Moreover, we will outline a DevOps strategy, detailing
key methods for adopting and scaling these practices within organizations.
Lastly, we will discuss the benefits of DevOps, such as faster delivery,
better quality, and improved collaboration. We will also review the DevOps
process, focusing particularly on continuous integration and continuous
delivery (CI/CD).

Understanding DevOps
Many technical innovations have taken place and challenged the traditional
method of IT management. The technological advances and changes are
quite thoughtful and rapid, including multiple fields such as agile
methodology, DevOps, cloud, big data, artificial intelligence, data science,
and so on. An innovative approach will undoubtedly be rewarding and
provide maximum value for organizations. By adopting these technologies,
many organizations and institutions have already gotten on this journey
toward a more promising future.
Nowadays, DevOps has taken center stage in the SDLC, and it offers
process frameworks, including various tools to integrate all the phases of
the application life cycle and ensure they function as a unified unit.
It helps in aligning and automating the process across the development
phase, testing, deployment, and support. It promotes best practices such as
code repositories, build automation, continuous deployment, and so on.
The term was introduced in 2007-2009 by Patrick Debois, Gene Kim, and
John Willis. They projected DevOps as the combination of Development
(Dev) and Operations (Ops). As per them, bridging the gap between
Developers and Operations team members can deliver business values to
users more quickly and make a company more competitive in the market.
DevOps has received a lot of attention over the last ten years, mainly in
business enterprises, as most enterprises outsource a lot of their IT work.
They have to rely on software houses for development and system
integrators for operations (sometimes outsourced), which incurred costs.
Then, the business leaders started understanding that it is good to put
developers and operators under one roof. This way, development and
operations work in one team, on the same product. They build and run it.
This can reduce cost and the overall development time, along with
improving the quality of the product.
DevOps is a mindset and a way of working that enables people and
organizations to develop and maintain sustainable work practices. It is a
cultural framework for developing a culture of collaboration, enabling the
team and its members to embrace their skills in effective and lasting ways.
It is a culture where we understand how we work and why.
Note: DevOps is innovating and adopting processes, culture, and
technology together in order to work more efficiently.

Figure 1.1 depicts the software development lifecycle:

Figure 1.1: Software development cycle

Also, the following figure shows the DevOps loop:

 Figure 1.2: DevOps loop

Many people think about DevOps as using specific tools like Ansible, Chef,
or Docker. However, what they forget is that tools alone cannot make a
product market ready in rapid time. The way we use tools is equally
important to make DevOps work. Along with using different tools, we need
to practice our skills, values, norms, and knowledge. Analyzing how a set
of people in teamwork, what technologies they use, and how it influences
the work can help us make important business decisions.
DevOps is based on three pillars:
• Collaboration: This is the very essence of DevOps—Rather than
teams separated by silos: development team, operations team,
testing team and so on., people from different teams are brought
under one roof having the same objective of delivering the product
with added value quickly and efficiently. This makes the DevOps
team a truly multidisciplinary team with better collaboration among
the team members. In one team, they can understand the
expectations in a better way.
• Processes: To perform the fast development and deployment, teams
must follow the entire development process in an iterative manner
that results in quality codes and quick feedback. These processes
should be integrated into the development workflow with
continuous integration and into the deployment workflow with
continuous delivery and deployment. The DevOps process is
divided into several phases:
○ Planning
 ○ Development
○ Continuous integration and delivery
○ Continuous deployment
○ Continuous monitoring
• Tools: Selecting the right tool and products used by teams is very
important in DevOps. When teams were separated into Dev and
Ops, each team used their specific tools according to their need, that
is, deployment tools for developers and infrastructure tools for Ops,
which often can lead to collaboration and communication gaps.
With multi-disciplinary teams that bring development and
operations together, the tools used must be uniform and can be used
by all members.
Development teams and operations teams both have to work in each other’s
areas to make a true DevOps team. Developers need to integrate monitoring
tools that the operations team uses to detect performance issues as earliest
and integrate with security tools provided by operations to protect access to
various resources.
Operations, on the other hand, should automate the creation and updating of
the infrastructure and use the code to achieve the infrastructure changes.
They have to integrate the code into a source code management tool; this is
called infrastructure as code, but this can only be done in cooperation
with developers who know what infrastructure is needed for applications.
Operations must also be integrated into application release processes.
The following figure illustrates the three blades of DevOps culture—the
collaboration between Dev and Ops, the processes, and the use of tools:
 Figure 1.3: Pillars of DevOps

The need for DevOps

Let us now discuss the reasons behind an increasing demand and need for
DevOps:

Business need
DevOps implementation differs for multiple scenarios, with added benefits
as listed:
• Decreased operational costs: While DevOps is praised for helping
organizations achieve continuous software development, it is also
highly esteemed by organizations for minimizing operational
expenditures by reducing costs involved in software development,
deployment, and maintenance.
• Improving client satisfaction: A noteworthy benefit of adopting
DevOps practices is that it drops the failure rate of new features and
improves the recovery time. The continuous deployment, testing and
constant feedback loops ensure quick service delivery and happier
customers. Teams can focus on building best products using
automation in the software pipeline and improve the business
 delivery. Implementing DevOps gives confident that the company
can stay ahead of the competition in the market.
• Maximizes efficiency and optimizes productivity: DevOps is
based on automation that lifts the productivity of teams and
promotes a performance-oriented culture. To solve all the recurring
problems, instead of waiting to act, automation manages the
repetitive tasks and employees focus on more productive and value-
added tasks.

Note: The 2017 State of DevOps Report quantifies the increase in

efficiency and reveals us that high-performing organizations
following DevOps practices spend 21% less time on unplanned work
and rework, and 44% more time on new work.6 In general,
successfully implementing DevOps practices can have a massive
impact on your organization by way of improvement in efficiency
and execution in areas that are both crucial and absolutely dull.
Source: [Link]
reports#2017

• Improving the software security: Integrating the security practices

from the beginning of the development cycle means security bugs
and other vulnerabilities can be addressed more easily rather than
fixing them at the deployment stage, which is more difficult and
costlier to fix. Integrating security into the development pipeline is
known as DevSecOps. Combining the development, security, and
operation across products and services improves the security and
quality of products along with quick delivery. This shift-left
approach (integrating security practices early in the development
life cycle) addresses the security vulnerabilities easily and almost
unnoticed to end users. This also ensures that outdated security
practices are not integrated into your software and do not interfere
with customer demand. So, if customer satisfaction and delivering
the high quality and secure products too quickly is your priority,
 then implementing a DevOps and DevSecOps process can save IT
headaches and deliver better business outcomes.
• Automation of software development cycle: Business needs are
met quickly with more automation and minimal manual
intervention, resulting in better quality products. Developers can
develop a product with their choice of open tools; the QA team can
create a QA system as a replica and deploy it to production
flawlessly and quickly.
• Single version of truth - source code management: There are
multiple versions of the code and it is difficult to determine the
appropriate code for the purpose. We lack a single version of the
truth. Code review feedback is through emails and not recorded,
leading to confusion and rework. DevOps advocates reliable source
code management and automated code reviews.
• Reliable configuration management: Building the code, testing,
and deployment happen on different systems. Sometimes, the code
works fine on the build system but does not work on production,
which sometimes causes delayed delivery and cost overheads.
Validating the different platforms and compatibility of dependencies
is manual and error prone. It is challenging to ensure that the code
works the same on all the systems having the same versions of the
tools, compilers, and so on. Automated and reliable configuration
management in DevOps ensures that the system is built the same for
different environments.
• Product readiness to markets: Even though we have a defined
process to develop code, test, and build it in defined timelines, there
are many various validations and manual checks in the process. The
integrations between different stages can also cause our
unpredictable delivery times. If we know how close our product is
to delivery and what its quality is, then we can plan the response in
advance rather than being reactive. Implementing DevOps shortens
the life development life cycle processes of product and makes it
ready to market.
 Automation of manual processes: Following manual processes is
• often error prone and repetitive. Using automation wherever
possible enhances efficiency. Testing cycle automation and
integrating with the build cycle, infrastructure service automation
such as creating, starting, stopping, deleting, terminating, and
restarting virtual machines will accelerate the efficiency and results
in better quality product.
• Containers: Portability of code is the main challenge. The code
works in development and QA environments, but deploying to
production systems causes multiple challenges such as code not
compiling due to dependency issues, build breakdown, and so on.
Building a code that works on all platforms is a challenge, and
maintaining multiple platform versions of development and QA is a
huge overhead. A portable container code would lessen these kinds
of issues.
• Other on-premise challenges: From capacity planning to
turnaround time, there are multiple challenges of having multiple
on-premise systems. The operational expenses are unpredictable.
Cloud migration gives the freedom to have multiple choices and
vendors. So there needs to be an efficient adoption method to ensure
results, and DevOps helps in achieving this.

Application development need

We will now discuss a few scenarios of application development needs:

Scenario 1
Assume a team was working on this large project with multiple applications
integrated to provide end-to-end functionalities. The launch date for a major
functionality was close when the customer asked to accommodate new
changes due to new government rules and regulations that had been
declared for the industry. So, the next iteration saw the team working on
this new piece of code changes. Due to ad-hoc changes to the code, a lot of
code got broken, and bugs got introduced.
Then there was another challenge, in order to incorporate even minor
feedback suggested by the customer, you have to follow the complete all
software development stages included development/code changes, code
review, unit testing, functional testing, regression testing and testing on
staging servers before it could be made ready to be shown to customer. So,
a change requiring just a few hours of development time would usually be
available for customer feedback on staging after 2 to 3 days and not
acceptable to the customer as it delayed the feedback cycle and launch.
So, the question is how to deliver quickly without compromising on the
quality.
Let us analyze the possible causes and why DevOps will be needed.
1. Testing cycles
Problem statement: Minimal unit testing was done, and that was
restricted to the new fix only. There were no sanity unit test cases,
which ensured that the new fix had not broken an existing
functionality. So, after deployment, additional issues were raised.
Solution: Writing automated unit test cases ensures that all related
test cases can also be run with the new change.
Problem statement: Due to complex integrations between
applications, manually running integration and regression testing
took the maximum time before functionality could be deployed to
staging.
Solution: Regression testing suite to include automated test cases.
2. Performance issues
Problem statement: Coding guidelines were not being followed
properly, and manual reviews sometimes missed the code
optimization. Which results in issues related to response time,
application crashes due to memory leaks, and so on.
Solution: Static analysis and code review tools like, Checkmarx and
Sonar to be used to maintain code consistency and code quality.
3. Task prioritization
Problem statement: Once the developer handovers the changes to
QA team, he moves on to the next development task. When the QA
team logs a bug, developers dispense their attention to work on a
new functionality and to resolve the existing bug. They were too
involved in working on the new functionality that most of the time,
they would not even have checked the new bugs logged in their
name. Similarly, after fixing the bug, developers have to do manual
entry in a tool to mark a bug as fixed so that it can be discussed
during standup calls. However, many hours get wasted before
everyone is updated on the status.
Solution: With continuous delivery pipelines, the developer would
get the feedback just a few minutes after checking in the code, and it
is a big-time saver!
4. Infrastructure, deployments and data issues
Problem statement: Many times, even after code working fine on
development servers do not work on QA servers and QA team raises
defects. Then, it will require a lot of effort to analyze the actual
problem. The problem could be related to configuration,
deployment, data issues, or any other reasons.
Solution: The preceding issues can be addressed by following an
infrastructure as a code mechanism to configure the server and make
an automated code-based deployment on all the environments,
which can be used for production as well. It was also recommended
to create environments like dev, test, and staging servers using
containers and destroy them once the task was completed. This
should be achieved with infrastructure as code so that the same code
can be reused to configure infrastructure and make deployments
across environments.
As you go through the solutions, it is clear that the answer to all the
above concerns lies in automating the entire process to improve
productivity and quality.
 All this can be achieved by DevOps pipeline that will have
automated unit tests, static analysis, automated functional tests, a
build triggered on checking code in source code version control,
infrastructure as a code, automated deployment, and an automated
pipeline to connect all these.

Scenario 2
Once a developer team completes all the coding and testing for a system, it
needs to be placed into an environment where:
• Only authorized people have access to it.
• It is compatible with all other systems and components with which it
interacts in the environment.
• It has sufficient resources on which to operate.
• The data that it uses should be up to date.
• The data that it generates should be usable by other systems in the
environment.
Furthermore, customer care personnel need to be trained in features of the
new system and operations personnel need to be trained in troubleshooting
any problems that might occur while the system is active and running. Now,
each of these items requires good coordination between the developers
and the operations team. Imagine a scenario where the functionality of the
system is not communicated properly by the development personnel to the
operations personnel. When coordination does not always happen in an
appropriate manner, system can’t work efficiently.
This type of problem is well addressed by the adoption of DevOps.
Automation of different levels reflects a well-defined release process that
persists beyond a single release and provides more than ever coordination
between development and operations.

DevOps and automation

The entire software development phase can be automated to some degree.
Only automation would not guarantee a successful DevOps practice but it is
necessary not only for technical benefits like scalability, reliability, quality,
and security but also for high deliverability impact on people. Any team
that is good at automation shows a higher level of performance.

As per the State-of-DevOps-Report-20211:

• 90% of respondents with highly evolved DevOps practices report
their team has automated most repetitive tasks.
• 97% of respondents with highly evolved DevOps practices agree
that automation improves the quality of their work.
• 62% of organizations stuck in mid-evolution report high levels of
automation.

Figure 1.4: State of automation in the team

The steps from build and testing through deployment can all be automated.
We will discuss the tools used in each one of these steps in coming
chapters, but here, we highlight the qualities of automation. Tools can
perform the actions required in each step of the process, starting from
building the code, checking the validity of code, testing the system,
providing feedback to appropriate personnel if errors occur in the process,
and maintaining a log for quality control, reporting, and auditing purposes.
Once tools become central to a set of processes, then the use of these tools
must also be managed. Tools are invoked, for example, from scripts,
configuration changes, or from the console. Where console commands are
complicated to use, it is advisable to create a script with their usage. Tools
may be controlled through specification files, such as Chef cookbooks,
Amazon CloudFormation, and so on. Similar to application code, the
scripts, configuration files, and specification files must be subjected to the
same quality control. The scripts and files should also be under version
control. This is termed infrastructure-as-code.
DevOps is not just automation, but automation is required. Automating the
repetitive task enables us to focus on deeper structural improvements, and
to shift our focus to more important deliverables in the team.

DevOps team structure

DevOps team structure plays a critical role in fully leveraging DevOps
benefits. As such, organizations should ensure that the team is built with the
right people with a clear definition of DevOps roles and responsibilities.
DevOps teams comprise professionals from the development, quality,
security, and operations division. The smallest DevOps team should
comprise the following people; A software developer/tester, automation
engineer/automation expert, quality assurance professional, security
engineer, and release manager. One of the advantages of a small team is that
they can make decisions quickly. Fewer members means fewer opinions
expressed in team meetings, resulting in quicker decision-making than with
a large team.
With infrastructure as code increasingly gaining momentum, the thin line
between development and operations is fading off. The current DevOps
team structure contains people who are skilled in coding and operations
both. Strong communication skills, technical expertise, and team player
mentality are important traits for a DevOps person.

Team roles
In this section, the roles within a development team with DevOps
responsibilities are discussed:
• Team lead: This role is responsible for facilitating the team,
obtaining resources for it, and protecting it from problems. This is
also called Scrum Master in Scrum or team coach or project lead
in other methods. This role embraces the soft skills of project
management but not the technical ones such as planning and
scheduling.
• Team member: This role, also known as developer or programmer,
is responsible for the creation and delivery of a system, services, and
product. This includes programming, testing, and release activities,
as well as other tasks related to software release.
Additional roles in a team executing a DevOps process include service
owner, reliability engineer, and DevOps engineer. An individual can
perform multiple roles, and roles can be split among individuals. The
assignment of roles to individuals depends on their skills and the workload
required to satisfy the role. Let us have a look at these roles in detail:
• Service owner: The service owner is responsible for outside team
coordination and is the single point of accountability in front of the
customer for all aspects of dedicated services. The ability to
communicate both with other stakeholders and with other members
of the team is the main role of the service owner. Another role is to
create the service life cycle roadmap so that it aligns with the vision
created by the business owner.
• Reliability engineer: Reliability engineers identify and manage
risks that can affect asset reliability and business operations. They
focus on risk management, loss elimination, and life cycle asset
management. Reliability engineers aim to reduce the cost of failure
by minimizing system downtimes. They ensure that the system
operations run effectively and efficiently. These types of engineers
use automation tools to solve problems by creating scalable and
reliable software systems. Standardization and automation are at the
heart of the reliability engineer role. They often have a software
 engineering background or system engineering or system
administration with IT operations experience.
The reliability engineer has several responsibilities:
○ The reliability engineer monitors the service in the time period
immediately after the deployment.
○ The reliability engineer is the point of contact for problems
with the service during its execution. This means being on call
for services that require high availability. Google calls this role
Site Reliability Engineer.
○ Once a problem occurs, the reliability engineer performs short-
term analysis to diagnose, mitigate, and repair the problem,
usually with the assistance of automated tools.
○ The reliability engineer has to be excellent at troubleshooting
and diagnosis. He also has to have a comprehensive knowledge
of the service so that a fix or workaround can be applied.
○ Increasingly, reliability engineers need to be competent
developers, as they need to write high-quality programs to
automate the repetitive part of the diagnosis, mitigation, and
repair.
• DevOps engineer: DevOps experts must have interpersonal skills
since they work across company silos to foster a collaborative work
environment. Besides having a strong understanding of common
system architecture, provisioning, and administration, DevOps
experts should also have experience with traditional developer
toolsets and practices, including source control, code reviews,
writing unit tests, and understanding agile principles.
In a DevOps way of software development processes, some of the tools
used are code testing tools, configuration management tools, continuous
integration tools, deployment tools, or post-deployment testing tools. Tools
require specialized knowledge and specialized input. The DevOps engineer
role is responsible for the care and feeding of the various tools used in the
DevOps pipeline. The DevOps engineering role is inherent in automating
the development and deployment pipeline. They are IT engineers with a
wide range of experience in development and operations, including coding,
infrastructure management, system administration, and DevOps pipelines.

Figure 1.5: Role and responsibilities of DevOps engineer

Responsibilities of a DevOps engineer include:

• Designing, building, testing, and maintaining the continuous
integration and continuous delivery (CI/CD) process.
• Analyzing and choosing the best tools and technologies for DevOps
implementation.
• Automating different phases of the DevOps pipeline.
• System monitoring.
• Managing the IT infrastructure.
• Ensuring system availability.
• Identifying application security measures by continuously
performing vulnerability assessments and risk management
(DevSecOps).

DevOps strategy
A good DevOps strategy helps the user gain a thorough and broader
understanding of its subject and its functions, multiple technologies, and its
various interfaces. A good strategy also provides organization a common,
focused and unbiased view of the current problems, how to develop the
future state, uncovers opportunities for growth, and results in enhanced
business outputs.
While planning a holistic DevOps strategy, one should ask most basic
questions:
• What are our business goals?
• What are we trying to achieve?
• How do our stakeholders see the value?
• What is the timeline for this?
• How do we plan the roadmap?
• What is the impact to the current business?
• How should we allocate our efforts?
• What are the benefits of doing it?
• What are the costs incurred by doing it?
An efficient and sustainable DevOps strategy for an organization will bring
multiple benefits, like channeling teams’ energy to focus on high-impact
problems, giving clarity to develop the future state of the system,
identifying growth opportunities, and creating ways for better business
outputs.
Planning a DevOps strategy will be a unique and extensive exercise. It
should cover every aspect of the software life cycle and focus on integrating
multiple technologies, platforms, and tools. Sometimes, various challenges
occur while planning a DevOps strategy that need to be handled with the
right skills and experiences.
An organization can consider the introduction of DevOps to fulfill specific
purposes, such as the following:
• Automating infrastructure and workflows (configuration
management)
• Automating code repositories, builds, testing
• Continuous integration and deployment
• Virtualization, containerization, and load balancing
• Big data and machine learning projects
• Automation of daily repetitive manual work
There is a wide range of open-source tools to select for adoption in specific
sections of DevOps, such as the following:
• GitHub: Git is a popular open-source version control tool. It is a
web-based service that helps developers store and manage their
codes in Git repositories. GitHub allows you to track and control
changes to your code.
• Jenkins: It is an open-source automation server that allows
continuous build, deployment, and testing, and is integrated with
various build tools such as Ant/Maven and the source code
repository Git. By using Jenkins, we can automate the end-to-end
development life cycle.
• Ansible: It is the tool that automates software provisioning,
configuration management, and application deployment with
agentless, Secured Shell (SSH) mode. Playbooks are the
mechanisms to create a Yum script file.
• SonarQube: It is the leading tool for continuously inspecting the
code quality and security of codebases and guiding development
teams during code reviews.
• Checkmarx: It provides static and interactive application security
testing, software composition analysis, infrastructure as code
security testing, and application security
 • Docker: A Docker is an open-source tool for automating the
deployment of applications as portable self-sufficient containers that
you can run on cloud or on-premise systems. Docker enables you to
separate your applications from your infrastructure to deliver
software quickly. You can manage your infrastructure the same way
you manage your applications.
• Kubernetes: It is an open-source container orchestration system for
software deployment and scaling. It groups containers into logical
units for easy management and discovery, handles scheduling on
nodes, and actively manages workloads to ensure their state matches
users' declared intentions.
• Chef and Puppet: Chef and Puppet are agent-based pull
mechanisms for the deployment automation.
Popular open-source tools are listed here:
• Source code management: Git, Bitbucket, GitHub and Subversion.
• Build management: MSBuild, Maven, Ant, and Gradle.
• Testing tools: JUnit, QUnit and Selenium.
• Repository management: Nexus, Artifactory, and Docker hub.
• Continuous integration: Jenkins, Bamboo, and TeamCity.
• Configuration provisioning: Ansible, Chef, Puppet, and Salt.
• Release management: Octopus Deply, Serena Release, and
StackStorm.
• Cloud: AWS, GCP and Azure. ( best places to build and run open
source software )
• Deployment management: Code Deploy, Rapid Deploy and Elastic
box.
• Collaboration: Jira, Slack and Team Foundation.
• BI/Monitoring: Elasticsearch, Kibana, and Nagios.
• Logging: Splunk, datadog, Logstash, and Logentries.
 • Container: Docker, podman, and Microsoft Azure container
Registry.
• Container orchestration: Kubernetes

Benefits of DevOps
In this section, we will discuss some benefits of DevOps as described in
Figure 1.6:
• Clients centric: The move to DevOps is crucial because it makes
the team more client-centric. It is good to believe that the final goal
in software development is a good software but if the product’s
development takes long time, can it make software wonderful at the
end? Maybe. But we should not overlook the most critical factor:
the consumer. The customer wants a solution, a usable product that
will address their problem. They don’t care much about the process
but about delivering an excellent product that is too quickly. As we
focus on more minor releases and we receive fast feedback, DevOps
naturally makes you client-centric.
• Quick resolution time: The team with the continuous feedback
loop is the most successful team. This way, DevOps teams can
reduce downtime and handle issues faster. Customer satisfaction
plunges if critical issues aren’t fixed promptly. With the lack of
better collaboration and open communication, critical problems
often increase stress and dissatisfaction among teams. Better
communication and collaboration allow development and operations
teams to work together to swiftly resolve issues or incidents.
• Speedy delivery time: DevOps is an approach that works on
automation to assure a smooth SDLC flow. DevOps’ fundamental
principles, like automation, continuous delivery, and a fast feedback
cycle, aim to make software development swifter and more
effective. Its culture of collaboration makes it possible to receive
instant feedback, quick bug resolution, and quick release
completion.
• Team collaboration for quicker product shipments: Another
benefit of DevOps is that it allows development and operations
teams to work together in a collaborative way to accelerate the
delivery of software. Is it crucial for businesses to shorten the
development timeline? The answer is “Yes”. It is a competitive edge
if you can make product market ready rapidly while maintaining
good quality.

Figure 1.6: Benefits of DevOps

• Quicker deployment: Building new products more effectively by

incorporating quick feedback from developers, co-workers, and key
stakeholders is unquestionably beneficial for the DevOps strategy.
Thanks to the DevOps process, businesses stay afloat, resulting in
steady implementations. Now, IT businesses can deploy faster.
• Better management of surprises: Every team faces unplanned
work that most often impacts team productivity. With established
processes and clear prioritization, development and operations
teams can better manage unplanned work while continuing to focus
on planned work. Prioritizing and assigning unplanned work across
different teams and systems is tough and sometimes distracts from
the work at hand. However, through raised visibility and proactive
retrospection, teams can better anticipate and share unplanned work.
Teams who fully embrace DevOps practices handle unplanned work
smarter and faster, and deliver better quality to their customers. The
increased use of automation and cross-functional collaboration
reduces complexity and errors, which in turn improves the Mean
Time to Recovery (MTTR) when incidents and outages occur.
Switching to DevOps takes a lot of effort. It is all about shifting the teams’
mind-set from “finishing the given task” to “making the product or feature
ready to be deployed.” Before introducing DevOps, it is essential to plan
the transition satisfactorily. You should know all the benefits of DevOps,
which will make you able to deal with it remarkably.

DevOps process
Before we proceed and dive deep to understand the entire DevOps pipeline
in the coming chapters, let us go through the following DevOps processes
and what tools to use. The DevOps standard processes prescribed across the
industry and adopted by organizations are listed as follows:
• Source code management
• Source code review
• Configuration management
• Build management
• Artifacts repository management
• Release management
• Test automation
• Continuous integration
• Continuous delivery
• Continuous deployment
• Infrastructure as code
• Application performance monitoring

Source code management

Source code management (SCM) is used to track modifications to a
source code repository. It enables multiple developers to develop code
concurrently across multiple development centres, sometimes spread across
diverse geographies. SCM tracks a running history of changes to a code
base. It permits the maintenance of a single source of code across different
developers and helps with collaborating on a single software project where
different developers work on the same code base. It also helps resolve
conflicts when merging updates from multiple contributors. SCM is also
synonymous with Version control. Integrating them with DevOps processes
offers solid integration and automation. We will see this in detail in the
coming chapters.
Some open-source SCM tools are as follows:
• Git
• Mercurial
• Subversion (SVN)
• Concurrent Version System (CVS)

Source code review

Code reviews are an essential process to improve the quality of software.
Code reviews help identify and remove common vulnerabilities such as
buffer overflows, formatting errors, and memory leaks. The code review
process can be done through multiple methods, such as formal meetings and
interactions to review the code through code walkthroughs or tool-assisted
code reviews.
Code reviews promote better collaboration between software development
team members. Code defects are identified and corrected before the
integration, improving the code's overall quality.
Some open-source tools for code review automation:
• GitHub
• GitLab
• SonarQube
• Bitbucket
• Codebrag
 • Review board
• Gerrit

Configuration management
Configuration management is a system engineering process for maintaining
systems, servers, and software in a desired, consistent state. Software
configuration management is a system engineering process that tracks and
monitors changes to a software system’s configuration metadata. In
software development, configuration management is commonly used
alongside version control and CI/CD infrastructure.
The configuration management process includes identification, verification,
and maintenance of configuration items of a system, both software and
hardware, such as patches and versions. A configuration management tool
will validate the relevance of the configurations on the system as per the
requirements. A common example is ensuring that the piece of code
working effectively on a development system should effectively work on a
QA system and production system. Any configuration parameter loss
between the systems will be tragic for the application's performance.
As per DevOps, incorporating configuration management processes and
tools enables organizations to perform impact analysis due to the
configuration change. It allows automated environment provisioning for
dev, QA, and prod. It also expedites verification of the systems and if
required effectively manages new updates.
A few popular software configuration management tools are as follows:
• Ansible
• Chef
• Salt
• Puppet

Build management
Build management is the process of collecting all the components of a
software application, such as source code, compilers, and dependencies, to
compile and build the application software. Builds can be manual, on
demand, and automatic. On-demand automated builds are written on script
to launch the build. Scheduled automated builds are the ones where
continuous integration servers run the builds. Triggered automated builds in
a continuous integration server are automatically launched once the code is
committed to a Git repository.
A few build tools that are in use are as follows:
• MSbuild
• Maven
• Ant
• Gradle
• Visual Build
• Grunt

Artifacts repository management

In a usual CI/CD pipeline, a lot of builds are created on a regular basis. The
artifact repo is required to store all the builds at a common location. A build
artifacts repository manager is a dedicated server for hosting multiple
repositories of binary components of successful builds and their
dependencies.
The benefits of artifacts repository management:
• Manage artifact life cycles and ensure that builds are repeatable and
reproducible.
• High availability of artifacts with access controls that can be shared
across teams and vendors.
• Define retention policies based on artifacts for audit compliance.
A few repository tools that are in use are as follows:
 • Sonatype Nexus
• NuGet
• JFrog Artifactory
• Docker hub

Release management
Release management is the process of managing, planning, scheduling, and
controlling a software build through different stages and environments; it
includes testing and deploying software releases.
This process facilitates a release's movement from development, testing,
and deployment to support and maintenance. It interfaces with several other
DevOps process phases in the SDLC. Release management has been an
integral part of the development process for years. However, its inclusion
into the DevOps process makes a complete cycle for automation.

Figure 1.7: Release management

From the DevOps perspective, release management is an iterative cycle

starting with a request for the new feature addition or changes in the
existing functionality. Once the proposed change is approved, the new
version is designed, built, tested, reviewed, and, after acceptance, deployed
to production. After deployment to production, the support phase starts.
During the support phase, there could be a possibility of functional
enhancement or performance enhancement leading to the beginning of a
new development cycle.
A few release management tools are:
• MS Visual Studio
• Octopus Deploy
• Continuum
• Electric Cloud
• Quikbuild
• Automic
• BMC Release Process Management
• Rally

Test automation
Manual testing for every possible scenario is tiresome, expensive, and time-
consuming. Test automation, or automatic testing, is running test cases
without manual intervention. Test cases are run with an automation tool or
through scheduled automation scripts. Though not all test cases qualify to
be run automatically, the majority of them can be scheduled.
Adopting test automation improves the effectiveness of the overall testing
life cycle, which results in improved software quality. Manual testing
efforts are replaced with automation, giving quick turnaround time.
A few test automation tools are as follows:
• Visual Studio Test Professional
• Selenium
• QTP (UFT)
• TestDrive
• SoapUI
Continuous integration
Continuous integration is a DevOps practice wherein multiple developers
continuously integrate their code changes into the main source code
repository of a project. The advantage of such a process is the transparency
of the code’s quality and fitness for its anticipated purpose. We will discuss
this in detail in Chapter 5, Continuous Integration.
There are a few prerequisites to achieving continuous integration:
• Using a version control repository for source code.
• Regular code check.
• Automate testing.
• Automate the build.
• Deploy build in preproduction.
Some available continuous integration tools are as follows:
• Jenkins
• Bitbucket
• TeamCity
• Travis
• Microsoft Teamcenter
• Bamboo
• GitLab CI
• CircleCI

Continuous delivery
Continuous delivery is the subsequent step of continuous integration in the
software development cycle; it enables swift and reliable software
development and product delivery with the minimal manual effort.
In continuous integration, code is developed and reviewed, followed by
automated building and testing. In continuous delivery, the product is
moved to the preproduction (staging) environment where thorough testing
for user acceptance is performed, and software is ready to release. The final
deployment into production is a manual step based on timings and a
business decision.
With continuous delivery, developed code is continuously delivered with
maximum automation and minimal manual intervention. The tools for
continuous delivery are the same as those that perform continuous
integration.

Figure 1.8: Continuous delivery vs continuous deployment

Continuous deployment
Continuous deployment is the complete process cycle of code change,
passing through all the phases of the software life cycle from code commit
to production deployment.
With continuous deployment, you automate the entire process. This is also
termed as automated application release--through all stages, such as the
code compilation, ensuring the dependencies are integrated, deployments
done once all the validation and testing passes.
With continuous deployments product releases with the code change are
accelerated with automation. It promotes effective collaboration between
Dev, QA, and operation teams which leads to higher output and better
customer satisfaction. We will see more in detail in Chapter 9, Continuous
Delivery and Deployment.

Infrastructure as code
Infrastructure as code (IaC) is a key DevOps practice that involves the
management of infrastructure, such as networks, compute services,
databases, storages, and connection topology, in a descriptive model. IaC
allows teams to develop and release changes faster and with greater
confidence.
IaC is a key DevOps practice means to manage infrastructure services, such
as networks, compute services, databases, and storage through the
configuration files. In DevOps' scope, IaC is the automation of routine tasks
through code, typically as configuration definition files, such as shell
scripts, Ansible playbooks, Puppet manifests or Chef Recipes.
It usually works on a server and client setup with push or pull-based
mechanisms, or agentless through an SSH. Many regular tasks on systems
such as creating, starting, stopping, deleting, terminating, and restarting
virtual machines are performed through software. The IaC shifts the manual
system administrative tasks performed on premise systems to automatic
which can be managed like software code. They are maintained in source
code repositories and also tested for deployment. IaC allows teams to
develop and release changes faster and with greater confidence.
Some IaC tools are as follows:
• Ansible tower
• Chef
• Puppet
• SaltStack
 • CFEngine

Application performance monitoring

Performance metrics are an essential part of every tool, service and product.
Organizations are more observant of the performance monitoring of their
applications, services and products. To achieve high-quality output for any
product, attaining a high degree of standard in process and performance
metrics is a prerequisite. Performance metrics can be measured on the basis
of multiple parameters to measure such as, for example, applications
availability, hardware systems availability, uptime versus downtime
responsiveness, acknowledgment, ticket categorization, resolution
timelines, and so on.
With DevOps continuous improvement processes to measure the metrics
and feedback, several tools are available for application monitoring;
A few tools are:
• Splunk
• DataDog
• AppDynamics
• AppNeta

Conclusion
In this chapter, we explored how the heart of DevOps begins with people
working not only as groups, but as teams with a desire for mutual
understanding and collaboration. These teams work together, communicate
their goals and challenges, and dynamically adjust in order to work toward
their shared organizational goals. We also discussed why we need DevOps.
Then, we outlined different steps of the CI/CD process and explained the
benefits of implementing DevOps.
In the next chapter, we will analyze the Monolithic architecture, understand
the needs of its migration to Microservices, and how to effectively plan the
migration.

1. Source: [Link]
s#2021

Join our book’s Discord space

Join the book’s Discord Workspace for Latest updates, Offers, Tech
happenings around the world, New Release and Sessions with the Authors:
[Link]
 CHAPTER 2
Slicing the Monolith

Introduction
Throughout history, people have loved breaking down complicated ideas
into simpler parts. This approach, a mix of analysis and synthesis, helps us
understand things better. Even Aristotle praised this method, calling it the
breakdown of complex things into simple elements.
In the world of making software, we do something similar. We take systems
apart, figure out what goes in and what comes out, and recognize certain
functions that are always needed. These common functions suggest that
having reusable and well-defined parts could make building software easier,
letting developers focus more on the unique stuff.
Microservices, a popular way of organizing software, has become a big deal
lately. It promises great benefits, but we need to avoid common mistakes,
follow best practices, and understand the key ideas to get those. There is
also the comparison between big, all-in-one systems (Monolithic) and a
more modular, interconnected approach of Microservices. Understanding
the nuances between Monolithic and Microservices-based architectures is
crucial. This chapter embarks on a comprehensive exploration, uncovering
the fundamental differences, strategic insights, and step-by-step guidelines
of these architectural paradigms.

Structure
In this chapter, we will discuss the following topics:
• Monolithic vs. Microservices based architecture
• The need for Microservices
• Planning Monolithic to Microservices migration
• DevOps and Microservices
• Microservices frameworks
• Serverless

Objectives
By the end of this chapter, you will have explored Microservices, compared
them with Monolithic systems, and discovered when to use each. We will
guide you through analyzing and upgrading from Monolithic to
Microservices setups and explore the collaboration between DevOps
practices and Microservices. You will also get hands-on experience with
popular Microservices frameworks like Spring Boot, Micronaut, and
Quarkus, and learn about serverless computing using AWS Lambda with
Quarkus. By the end, you will also have gained a practical understanding of
different architectural approaches and modern development tools.

Monolithic vs. Microservices based architecture

In the process of software development, the architectural choices made can
significantly impact the scalability, maintainability, and overall performance
of an application. Two prevalent approaches are Monolithic architecture and
Microservices architecture, each with its advantages and challenges. Let us
look at both approaches in detail.

Monolithic architecture
Monolithic architecture refers to the traditional and unified structure where
all components of an application are tightly interconnected and run as a
single unit. In a Monolithic application, the entire codebase, including the
user interface (UI), business rules, and data accessing layer, is combined
into a single, cohesive unit.
Some characteristics of Monolithic architecture are as follows:
• Single codebase: All components are part of a singular codebase,
making it easier for developers to navigate and manage the
application.
• Tight integration: Components are tightly integrated, facilitating
seamless communication between different application parts.
• Simplicity: Monolithic architectures are often considered simpler to
develop and deploy, especially for smaller applications with
straightforward requirements.
• Scaling challenges: Scaling a Monolithic application involves
replicating the entire system, which might be inefficient for specific
components that require more resources.

Microservices architecture
Microservices architecture, on the other hand, is a more modern and
distributed approach where an application is divided into smaller,
autonomous services that interact with each other via APIs. Each service
manages a distinct business capability and can undergo development,
deployment, and scaling independently.
Some characteristics of Microservices-based architecture are as follows:
• Service independence: Microservices operate as independent
entities, each responsible for a specific business function. This
fosters flexibility and allows for independent development,
deployment, and scaling.
• Distributed communication: Services communicate via APIs,
enabling seamless integration while maintaining autonomy. This
 distributed nature contributes to fault isolation and improved system
resilience.
• Scalability: Microservices offer granular scalability, allowing
organizations to scale individual services based on demand rather
than scaling the entire application.
• Technology diversity: Different services within a Microservices
architecture can be developed using varied technologies, allowing
teams to choose the best tools for specific tasks.

The need for Microservices

In software development, the choice between Monolithic architecture and
the emerging Microservices paradigm has become crucial for making
scalable and resilient applications. Let us take a journey to understand
Monolithic applications and why the software industry is moving towards
Microservices. Additionally, we will explore the essential steps involved in
planning a migration from a Monolithic to a Microservices-based
architecture. However, first, take a look at the following example of
Monolithic architecture:
Consider a traditional e-commerce application where the user interface,
order processing, and inventory management are tightly coupled within a
Monolithic structure. As the business grows, scaling becomes challenging
because scaling one component means scaling the entire application,
including parts that may not require additional resources.
Now, we will analyze the above Monolithic application on different
parameters:
• Simplicity and development speed: Monolithic architectures are
often praised for their simplicity. Developers can work on the entire
application without worrying about communication between
different components.
• Testing and debugging: Testing a Monolithic application can be
straightforward since all components are tightly integrated.
 However, debugging can be challenging as the codebase grows.
• Scalability: Scaling a Monolithic application usually involves
replicating the entire system, which might not be efficient for
specific components that require more resources.
• Dependency management: Changes to one part of the code may
have cascading effects, making it crucial to manage dependencies
carefully.
To address various challenges of Monolithic applications and why we
should move to Microservices, let us analyze:
• Scalability challenges: Monolithic applications may face
challenges when trying to scale specific features independently.
Microservices offer a solution by allowing selective scaling.
• Improved fault tolerance: Microservices provide better fault
isolation, reducing the impact of failures and enhancing the system's
overall reliability.
• Technology heterogeneity: As applications grow, different
components may require different technologies. Microservices
accommodate this diversity, allowing teams to choose the best tools
for each service.
• Continuous deployment: Microservices facilitate continuous
deployment, enabling faster release cycles and reducing the time it
takes to bring new features to users.
• Team autonomy: Microservices enable teams to work on
independent services, promoting autonomy and faster development
cycles.
We will now look at an example of Microservices-based architecture:
In the e-commerce scenario we discussed earlier, imagine adopting a
Microservices approach. Each business function—user authentication, order
processing, payment handling, and inventory management—becomes an
independent Microservice. This autonomy allows teams to develop, deploy,
and scale each service independently, offering a more agile and scalable
solution.
The choice between Monolithic and Microservices architectures depends on
the specific requirements and goals of a project. Monolithic architectures
are simpler to develop and maintain but may face challenges as applications
grow. With their distributed nature, Microservices provide solutions to
scalability and flexibility issues, making them a popular choice for modern,
dynamic applications. Ultimately, the decision should align with the
project's needs and the organization's long-term goals.

Planning Monolithic to Microservices migration

Transitioning from Monolithic to Microservices architecture is a key
strategy for achieving scalability and agility. It requires meticulous planning
and strategic considerations. To navigate this shift efficiently, you can
follow the underlying baseline guidelines for a systematic and practical
approach:
• Understanding the starting point: Assessing the monolith
○ Thorough assessment: Begin by deeply understanding your
existing Monolithic setup. Identify dependencies and potential
challenges.
○ Scalability check: Evaluate how well your current system can
scale. Identify any bottlenecks that might hinder growth.
○ Business priorities: Understand the business impact of the
migration. Prioritize essential functionalities based on business
value.
• Setting Microservices boundaries: Clearing the lines
○ Functional breakdown: Breakdown your Monolithic
application into smaller, manageable services. Define the scope
of each Microservice based on specific functions.
○ API clarity: Clearly define how Microservices will
communicate using APIs. Ensure these interfaces are well-
 documented and standardized.
○ Dependency map: Map out dependencies between
Microservices. Understand how data flows and interactions
occur within the system.
• Gradual decomposition: A step-by-step approach
○ Smart prioritization: Prioritize which parts of the monolith to
decompose first based on impact and dependencies.
○ Incremental changes: Make changes gradually to minimize
disruption. Test and release Microservices in small increments.
○ Feedback loop: Create a feedback loop with teams and users.
Use feedback to refine your migration strategy.
• Data management: Ensuring consistency
○ Data strategies: Address data challenges during migration.
Consider using Microservices databases (database-per-service)
or event sourcing (use event-driven systems like Apache
Kafka, Rabbit MQ, etc.) for consistent data.
○ Transaction integrity: Implement mechanisms to maintain
data integrity across Microservices. Consider models of
eventual consistency or Saga patterns.
• Implementing DevOps practices: Automation and collaboration
○ Automate deployment: Embrace DevOps for automated
deployment, testing, and monitoring. Use continuous
integration and deployment pipelines for efficiency.
○ Collaboration culture: Encourage collaboration between
development and operations teams. Foster a culture of
teamwork for a smoother transition.
• Monitoring and debugging: Ensuring operational excellence
○ Monitoring tools: Set up monitoring systems to track
Microservices' performance. Use real-time monitoring and
logging to catch issues early.
 ○ Effective debugging: Provide tools and strategies for effective
debugging. Equip teams to troubleshoot and resolve issues
within Microservices.
• Team empowerment: Nurturing autonomy
○ Skills enhancement: Train teams on the new Microservices
architecture. Ensure they have the skills needed for
independent development and deployment.
○ Autonomy and ownership: Foster a culture of autonomy
within Microservices teams. Encourage ownership and
accountability for their services.
• Continuous monitoring and optimization: Fine-tuning the system
○ Regular monitoring: Continuously monitor Microservices'
performance. Use metrics to identify areas for improvement.
○ Optimization strategies: Optimize based on real-world usage
and feedback. Refine and enhance the Microservices
architecture iteratively.
• Documentation and communication: Ensuring clarity
○ Comprehensive documentation: Document the new
architecture thoroughly. Include details on Microservices
boundaries, APIs, dependencies, and data flows.
○ Stakeholder updates: Communicate changes to stakeholders.
Keep everyone informed and aligned with organizational goals
throughout the migration.
Transitioning from Monolithic to Microservices architecture is a complex
but rewarding journey. Above simplified guide, encompassing assessment,
gradual decomposition, DevOps practices, and continuous optimization,
aims to make this shift more manageable. By fostering a collaborative and
autonomous culture, organizations can unlock the benefits of Microservices
and adapt to the ever-changing landscape of software development.

DevOps and Microservices

When we talk about the continuously changing world of application
development, two transformative paradigms—DevOps and Microservices
—have emerged as pillars of modern application architecture. DevOps,
emphasizing collaboration and automation across development and
operations teams, seamlessly aligns with the principles of Microservices, a
modular and scalable architectural style. In this section, we will explore
how Microservices fit perfectly within the DevOps philosophy, creating a
powerful synergy that propels organizations toward enhanced agility, rapid
deployment, and continuous innovation.

Understanding DevOps and Microservices

Before we discuss the integration of DevOps principles with Microservices,
which unlocks the potential of streamlined and agile software development,
let us first explore the fundamentals of DevOps and Microservices:

DevOps is the cultural shift

DevOps is more than just a set of practices. It is a cultural shift that unites
development and operations teams to work collaboratively, break down
silos, and streamline the software delivery lifecycle. The core tenets of
DevOps include automation, continuous integration, continuous delivery
(CI/CD), and a focus on culture, fostering a shared responsibility for the
entire application lifecycle.

Microservices is the modular approach

Contrastingly, the Microservices architectural style involves structuring an
application into a series of loosely linked, independently deployable
services. Each of these services, known as Microservices, is tailored to
carry out specific business functions, and they communicate with one
another through APIs.
This modular approach enables scalability, flexibility, and ease of
maintenance.
The synergy of DevOps and Microservices
The integration of DevOps principles with Microservices presents a
dynamic approach to meet the demands of the modern tech landscape. This
synergy leverages the modular nature of Microservices, aligning seamlessly
with the continuous integration, deployment, and iterative improvement
philosophy championed by DevOps methodologies. Let us dive into how
this combination of DevOps and Microservices works on the following
parameters:
• Rapid development and deployment: Microservices, with their
modular structure, align perfectly with DevOps principles of
continuous integration and continuous delivery. Development teams
can independently work on and deploy Microservices, allowing for
rapid and frequent releases.
• Agile development and iterative improvement: DevOps promotes
an agile development approach, and Microservices inherently
supports this by allowing teams to work on small, focused
components. This facilitates iterative development, enabling teams
to respond quickly to changing requirements and deliver
incremental improvements.
• Isolation and fault tolerance: Microservices' independent nature
allows for better fault isolation. If a single Microservice experiences
a failure, it does not result in the entire application going down. This
aligns with the DevOps principle of resilience, where the system
should gracefully handle failures without a catastrophic impact.
• Automated testing and deployment: DevOps places a strong
emphasis on automated testing and deployment, ensuring the
reliability and consistency of the software delivery process.
Microservices, being independently deployable, fit seamlessly into
automated testing and deployment pipelines, contributing to a more
efficient and reliable release process.
• Scalability and resource efficiency: Microservices support the
DevOps goal of scalability by allowing individual services to be
 scaled independently based on demand. This ensures optimal
resource utilization and the ability to adapt to changing workloads
without scaling the entire application.
• Monitoring and continuous improvement: DevOps encourages
continuous monitoring and feedback loops for performance
optimization. Microservices, with their independent operation,
facilitate granular monitoring, making it easier to identify and
address performance issues. This aligns with the DevOps principle
of continuous improvement.
Now, let us discuss the best practices for implementing DevOps and
Microservices:
• Containerization: Embrace containerization technologies like
Docker to encapsulate each Microservice and its dependencies. This
ensures consistency across different environments, easing the
deployment process.
• Orchestration: Utilize container orchestration tools like Kubernetes
to manage and scale Microservices effectively. Kubernetes
automates deployment, scaling, and operations, aligning with the
automation focus of DevOps.
• Infrastructure as code (IaC): Adopt IaC practices to automate the
provisioning and management of infrastructure. Tools like
Terraform enable the consistent and automated setup of
environments, promoting infrastructure reliability.
• Continuous integration and deployment: Implement robust
CI/CD pipelines for each Microservice. This ensures that changes
are automatically tested and deployed, following the continuous
delivery principles of DevOps.
• Monitoring and logging: Set up comprehensive monitoring and
logging for each Microservice. Centralized monitoring tools offer
visibility into the performance and health of the entire system,
supporting the DevOps goal of continuous feedback.
The following are some case studies of successful integration of DevOps
and Microservices:
• Netflix: Microservices at scale: Netflix, a pioneer in the streaming
industry, leverages a Microservices architecture supported by a
strong DevOps culture. Their ability to release new features rapidly
and handle a massive scale is a testament to the successful
integration of DevOps and Microservices.
• Amazon: DevOps and Microservices for flexibility: Amazon, with
its vast and diverse ecosystem, employs a Microservices architecture
with a strong DevOps foundation. This allows them to adapt quickly
to market changes, experiment with new features, and maintain high
availability across their services.
The fusion of DevOps and Microservices creates a harmonious ecosystem
where agility, scalability, and rapid innovation thrive. DevOps practices
seamlessly align with the principles of Microservices, providing
organizations with the framework to build, deploy, and scale modular
services efficiently. As businesses navigate the complexities of modern
software development, the symbiotic relationship between DevOps and
Microservices stands as a beacon, guiding the way towards a future where
continuous improvement and adaptability are at the forefront of
technological evolution.

Microservices frameworks
Navigating the vast JVM ecosystem reveals a plethora of alternatives
tailored for specific use cases. The abundance of Microservice frameworks
and libraries can make selecting the right one a challenging task.
Certain frameworks have garnered popularity due to various factors such as
developer experience, extensibility, time-to-market, resource consumption
(CPU, memory), start-up speed, failure recovery, documentation, and
integration with third-party tools. This section explores three prominent
candidates in the Microservices landscape: Spring Boot, Micronaut, and
Quarkus. Consider that certain instructions might need adjustments due to
the rapid evolution of these technologies. We highly advise reviewing the
documentation for each framework, as newer versions may introduce
changes.
Furthermore, it is essential to have Java 17 as a minimum for these
examples, and exploring Native Image additionally demands the installation
of GraalVM. There are diverse methods available for installing these
versions in your environment, and we suggest leveraging SDKMAN! for a
streamlined installation and management process. To keep things concise,
the focus is solely on production code, considering that delving into each
framework could be an extensive endeavor. It goes without saying that
thorough testing procedures should be implemented. The objective for each
example is to create a straightforward "Hello World" REST service capable
of receiving an optional name parameter and responding with a greeting.
Now, let us talk a little about GraalVM:
For those unfamiliar with GraalVM, it serves as an umbrella project
encompassing various technologies that enable distinctive features:
• Just-in-Time (JIT) compiler: GraalVM boasts a JIT compiler
implemented in Java, which dynamically compiles code on the fly.
Notably, Graal is the most modern JIT compiler, written entirely in
Java.
• Substrate VM: A virtual machine within GraalVM, Substrate VM
supports running hosted languages like JavaScript, Python, and R on
top of the Java Virtual Machine (JVM). This integration ensures
that hosted languages benefit from tighter integration with JVM
capabilities.
• Native image: This utility, relying on ahead-of-time (AOT)
compilation, converts bytecode into machine-executable code. The
output is a binary executable specific to the platform.
All three frameworks under consideration in this discussion support
GraalVM, utilizing GraalVM Native Image primarily to create platform-
specific binaries. The primary objective is to reduce deployment size and
enhance memory efficiency. It is important to recognize a trade-off between
employing Java mode and the GraalVM Native Image mode. While the
latter can yield binaries with a smaller memory footprint and quicker start-
up time but requires longer compilation time, long-running Java code will
progressively optimize, benefiting from one of the key features of the JVM.
Native binaries, on the other hand, cannot be optimized while they are
running. The development experience also fluctuates, necessitating
additional tools for tasks such as debugging, monitoring, and measurement.

Spring Boot
Among the three candidates, Spring Boot stands out as the most well-
known, building upon the legacy established by the Spring framework. The
majority of Java developers have encountered Spring-related projects,
making Spring Boot the favored choice.
In the Spring framework methodology, the approach centers around
constructing applications, specifically Microservices in our context, by
composing the pre-existing components. This method places a strong
emphasis on customizing configurations to ensure low-cost code ownership,
operating under the assumption that your specific logic is more concise than
the comprehensive features offered by the framework—a principle
applicable to many organizations. The trick is identifying an existing
component that can be fine-tuned and configured before considering the
development of a new one. The Spring Boot team actively integrates a
diverse range of useful components, including database drivers, monitoring
services, logging, journaling, batch processing, report generation, and more.
To initiate a Spring Boot project, the usual method involves visiting the
Spring Initializr website, where you can choose the specific features needed
for your application and then generate the project by clicking the Generate
button. This process generates a ZIP file you can download and use to
kickstart your development locally. In Figure 2.1, we have opted for the
Web and Spring Native features. The Web feature incorporates components
for exposing data through REST APIs, while the Spring Native feature
enhances the build by introducing an additional packaging mechanism
capable of creating Native Images with Graal.
After extracting the ZIP file and executing the ./mvnw verify command (for
windows, use mvnw verify) at the project's root directory ensures a solid
starting point. This command downloads dependencies, a standard Apache
Maven behavior for the first-time builds on the target environment.
Subsequent Maven commands will not trigger additional downloads unless
there are updates to dependency versions in the [Link] file.

Figure 2.1: Spring Initializer

A typical Spring Boot Microservices project structure looks like the

following:
├── [Link]
├── mvnw
├── [Link]
├── [Link]
└── src
├── main

│ ├
 │ ├── java
│ │ └── com
│ │ └── myproject
│ │ └── app
│ │ ├── [Link]
│ │ ├── [Link]
│ │ └── [Link]
│ └── resources
│ ├── [Link]
│ ├── static
│ └── templates
└── test
└── java
The project structure is organized with the necessary Maven files, a src
directory containing Java source code and resources, and a basic set of
classes, including [Link]. To expose data via REST in
JSON format, we need two additional classes: [Link] and
[Link]. You can create these two files using your
preferred text editor or IDE. This file [Link] should look
something like this:
package [Link];
public class MyGreeting {
private final String details;
public MyGreeting(String details) {
[Link] = details;
}
public String getDetails() {
return details;
}
}
[Link] defines a simple data object designed for rendering
content in JSON format. It is a straightforward, immutable class with a
content attribute. You might consider transitioning to a mutable
implementation, but for the present purpose, this will meet our needs.
Moving to the REST endpoint, defined as a GET request on the /greet path.
Spring Boot conventionally designates the controller stereotype for this type
of component, reflecting the historical preference for Spring model-view-
controller (MVC) in the development of web applications. You have the
flexibility to choose a different filename, but it is essential to preserve the
integrity of the component annotation. The second file
[Link] looks as follows:
package [Link];
import [Link];
import [Link];
import [Link];
@RestController
public class MyGreetingController {
private static final String DefaultMessage = "Hello, %s!";
@GetMapping("/greet")
public MyGreeting greet(@RequestParam(value = "name", defaultValue
= "Java") String name) {
return new MyGreeting([Link](DefaultMessage, name));
}
}
In the preceding code, [Link] defines a REST
endpoint as a GET call on the /greet path. The controller takes a name
parameter as input, defaulting to "Java" if not supplied. The return type is
the MyGreeting object, and Spring Boot automatically marshals data to
JSON based on annotations and defaults.
You can run the Spring Boot application using either ./mvnw spring-
boot:run or by generating the JAR and manually running it (./mvnw
package followed by java -jar target/[Link]). The
application will start an embedded web server on port 8080, with the /greet
path mapped to an instance of MyGreetingController. Test the API using
commands like:
$ curl [Link]
{“details”:”Hello, Java!”}
Or
$ curl [Link]
{“details”:”Hello, Microservs!”}
Now, to enhance the performance of the Microservices, we can leverage
GraalVM Native Image. The following steps guide you through the process:
1. Update [Link]: Modify the [Link] file to include the necessary
configurations for GraalVM Native Image:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="[Link]
xmlns:xsi="[Link]
xsi:schemaLocation="[Link]
[Link]
<modelVersion>4.0.0</modelVersion>
<parent>
 <groupId>[Link]</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>3.2.3</version>
</parent>
<groupId>[Link]</groupId>
<artifactId>app</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>app</name>
<description>Demo project for Spring Boot</description>
<properties>
<[Link]>17</[Link]>
<[Link]>0.10.0-SNAPSHOT</spring-native.v
ersion>
</properties>
<dependencies>
<dependency>
<groupId>[Link]</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>[Link]</groupId>
<artifactId>spring-native</artifactId>
<version>${[Link]}</version>
</dependency>
 <dependency>
<groupId>[Link]</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>[Link]</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<classifier>exec</classifier>
</configuration>
</plugin>
<plugin>
<groupId>[Link]</group
Id>
<artifactId>spring-aot-maven-plugin</artifactId>
<version>${[Link]}</version>
<executions>
<execution>
<id>test-generate</id>
<goals>
 <goal>test-generate</goal>
</goals>
</execution>
<execution>
<id>generate</id>
<goals>
<goal>generate</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>[Link]</groupId>
<artifactId>native-image-maven-plugin</artifactId>
<version>21.1.0</version>
<configuration>
<mainClass>
[Link]
</mainClass>
</configuration>
<executions>
<execution>
<goals>
<goal>native-image</goal>
 </goals>
<phase>package</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>spring-release</id>
<name>Spring release</name>
<url>[Link]
</repository
Install GraalVM: Ensure you have GraalVM installed with a
version matching the one in the [Link] file. Also, make sure the
native-image executable is installed using gu install native-image.
The gu command is a part of the GraalVM installation.
2. Generate native executable: Run the following command to
generate a native executable:
./mvnw -Pnative-image package
This process may take longer than usual, as it involves downloading
dependencies and generating a native image.
3. Run native executable: Once the build is complete, you will find a
new file, e.g., [Link], inside the
target directory. Run the native executable using the following
command:
./target/[Link]
 Notice the significantly faster start-up time compared to running in
Java mode. The trade-off is a larger executable size, but it provides a
single binary for running the application without requiring a Java
runtime.
Spring Boot, with its Microservices architecture, provides a robust
foundation for building scalable and modular applications. By integrating
GraalVM Native Image, developers can further optimize the performance
of their Microservices, achieving faster start-up times and reduced resource
consumption. We covered the basic setup and optimization steps, but Spring
Boot offers a rich set of features for building complex Microservices
architectures. Explore further to unlock the full potential of the framework
in your Microservices journey.

Micronaut
In 2017, Micronaut emerged as a modern reinterpretation of the Grails
framework, originally developed as a contemporary counterpart to Grails.
Grails, a successful clone of the Ruby on Rails (RoR) framework, utilized
the Groovy programming language. Despite its initial success, Grails faced
a decline in popularity with the ascent of Spring Boot, leading the Grails
team to explore alternatives, ultimately giving rise to Micronaut. Micronaut,
on the surface, delivers a user experience similar to Spring Boot,
empowering developers to construct applications through the integration of
existing components and thoughtful defaults.
Micronaut distinguishes itself by employing compile-time dependency
injection for building applications, diverging from the prevalent runtime
dependency injection method employed by Spring Boot to assemble
applications. This seemingly minor adjustment allows Micronaut to
optimize runtime performance, reducing application bootstrapping time and
resulting in lower memory usage. Additionally, it diminishes the
dependence on Java reflection, historically known for its slower speed
compared to direct method invocations.
Several methods exist for initiating a Micronaut project, with the
recommended approach being to browse Micronaut Launch. Here, you can
specify the desired settings and features to incorporate into your project.
The default application type establishes the minimum essential
configurations for building a REST-based application, similar to the one we
will walk through shortly. Once you have selected your preferences,
proceed to click the Generate Project button, illustrated in Figure 2.2. This
action produces a downloadable ZIP file that can be retrieved and utilized
in your local development environment.

Figure 2.2: Micronaut Launch

Similar to Spring Boot, initializing a Micronaut project involves extracting

the content of ZIP file and executing the ./mvnw verify command at the
project's root directory. This command smoothly handles plugin and
dependency downloads, ensuring a successful build within seconds. The
resulting project structure should look like the following, with additional
source files, embodying the essentials for building a REST-based
application:
├─── [Link]
├─── [Link]
├─── mvnw
├─── [Link]
├─── [Link]
├─── src
└── main
└── java
│ └── com
│ └── myproject
│ └── app
│ └── [Link]
│ └── [Link]
│ └── [Link]
└── resources
├─── [Link]
└── [Link]
The directory structure, including files like [Link] and [Link],
follows standard conventions. Noteworthy additions include the
[Link] file, reflecting Micronaut's commitment to simplicity.
The src directory, mirroring other Java projects, contains the main source
code, including [Link] defining the entry point and the
[Link] resource file retains its configuration properties.
Here, we have introduced two new source files: [Link], defining
a data object responsible for holding a message sent to the consumer, and
[Link], defining the actual REST endpoint. The
controller stereotype adheres to conventions established by Micronaut, with
the filename change allowed for your domain. The content for
[Link] is as follows:
package [Link];
import [Link];
@Introspected
public class MyGreeting {
private final String details;
public MyGreeting(String details) {
[Link] = details;
}
public String getDetails() {
return details;
}
}
The class follows an immutable design. The @Introspected annotation
instructs Micronaut to inspect the type during compile time, incorporating it
into the dependency injection procedure. In most cases, you can omit the
annotation, as Micronaut can deduce that the class is necessary. However,
its inclusion becomes crucial when generating the native executable using
GraalVM Native Image; without it, the executable will not be fully
functional. The second file [Link] looks as follows:
package [Link];
import [Link];
import [Link];
import [Link];
@Controller("/")
public class MyGreetingController {
private static final String DefaultMessage = "Hello, %s!";
@Get(uri = "/greet")
public MyGreeting greet(@QueryValue(value = "name",
defaultValue = "Java") String name) {
return new MyGreeting([Link](DefaultMessage, name));
}
}
The preceding defined controller defines a single endpoint mapped to
/greet, taking an optional parameter named "name" and returning an
instance of the data object. By default, Micronaut automatically convert the
return value into JSON format.
To run the application, use either ./mvnw mn:run to run it as part of the
build process or ./mvnw package to create a [Link] in the target
directory. This JAR file can be executed in standard manner using java -jar
target/[Link]. Test the API using commands like:
// employing the default name parameter
$ curl [Link]
{"details":"Hello, Java!"}
Or
// using a specific value for the name parameter
$ curl [Link]
{"details":"Hello, Microserv!"}
Each command launches the application rapidly. we can achieve an even
greater speed boost by transforming the application into a native executable
using GraalVM Native Image. Fortunately, the Micronaut approach is more
accommodating for this type of configuration, with everything we need
already set up in the generated project. That is it – no need to modify the
build file with additional configuration; it is all preconfigured.
For GraalVM Native Image transformation, ensure GraalVM and its native-
image executable are installed. Create a native executable with ./mvnw -
Dpackaging=native-image package. After sometimes, an executable
named "app" should be available in the target directory. When running the
application with the native executable, the result is a fast start up time at
least a one-third improvement in speed compared to Spring Boot.
Now, let us conclude our exploration of Micronaut and proceed to the next
framework, that is, Quarkus.

Quarkus
Quarkus, although officially introduced in early 2019, had its
developmental roots reaching back even further. Sharing similarities with
the previously discussed frameworks, Quarkus enhances the development
experience through its component-based approach, convention over
configuration, and productivity tools. Notably, Quarkus adopts compile-
time dependency injection, mirroring Micronaut's strategy and gaining
advantages like reduced binary size, quicker start-up times, and decreased
reliance on runtime intricacies. Beyond these shared attributes, Quarkus
introduces its distinctive features. Particularly noteworthy is its emphasis on
adhering to standards compared to its counterparts. Quarkus incorporates
the MicroProfile specifications, a set of standards developed by Eclipse
Foundation to address modern Microservices challenges, along with some
APIs originating from JakartaEE (formerly JavaEE).
To begin with Quarkus, navigate to Configure Your Application page,
where you can set values and download a ZIP file containing the necessary
configurations. The page offers numerous extensions for configuring
specific integrations like databases, REST capabilities, and monitoring.
Ensure to choose the RESTEasy Jackson extension, which allows Quarkus
to perfectly handle JSON marshaling. Refer to the following figure:

Figure 2.3: Quarkus “Configure Your Application” page

Upon clicking the Generate your application button, you will be prompted
to save a ZIP file locally, with contents resembling the following structure:
├── [Link]
├── mvnw
├──.dockerignore
├── [Link]
├── [Link]
└── src
├── main
│ ├── docker
│ │ ├── [Link]
│ │ ├── [Link]-jar
│ │ ├── [Link]
│ │ └
 │ │ └── [Link]-distroless
│ ├── java
│ │ └── com
│ │ └── myproject
│ │ └── app
│ │ ├── [Link]
│ │ └── [Link]
│ └── resources
│ ├── META-INF
│ │ └── resources
│ │ └── [Link]
│ └── [Link]
└── test
└── java
Quarkus, designed initially for Microservices and cloud architectures via
containers and Kubernetes, has expanded its capabilities to support various
application types and architectures over time. Quarkus generates the
[Link] file by default, which serves as a standard
JAX-RS (Jakarta RESTful Web Services) resource. However, to integrate
the [Link] data object, a few adjustments are required to that
resource. The following is the source code for these adjustments:
package [Link];
public class MyGreeting {
private final String details;
public MyGreeting(String details) {
[Link] = details;
}
public String getDetails() {
return details;
}
}
The code for the data object follows the same pattern we encountered
earlier in this chapter. There is nothing new about this immutable data
object. Now, regarding the JAX-RS resource, things are similar yet slightly
different. Despite seeking the same behavior as before, we instruct the
framework differently, using JAX-RS annotations. Therefore, the code
looks as follows:
package [Link];
import [Link];
import [Link];
import [Link];
import [Link];
@Path(«/greet»)
public class MyGreetingResource {
private static final String DefaultMessage = "Hello, %s!";
@GET
public MyGreeting greet(@QueryParam("name")
@DefaultValue("Java") String name) {
return new MyGreeting([Link](DefaultMessage, name));
}
}
If you are acquainted with JAX-RS, this code will not catch you off guard.
However, for those unfamiliar with JAX-RS annotations, here is the
breakdown: we label the resource with the REST path to which we want it
to respond. Additionally, we specify that the greet() method manages a
GET call, and its name parameter defaults to a specified value. There is no
need for further instructions to Quarkus; it automatically transforms the
return value into JSON.
Running the application can be done in several ways, and one notable
approach is leveraging Quarkus' developer mode during the build process.
This unique feature allows for effortless application execution,
automatically capturing and applying any adjustments without the need for
manual restarts. To activate this mode, simply use the command ./mvnw
compile quarkus:dev. If changes are made to the source files, the build
process autonomously recompiles and reloads the application.
You can alternatively execute the application using the Java interpreter, as
demonstrated earlier, with a command like java -jar target/quarkus-
app/[Link]. It is worth noting that we are using a different JAR,
even though the [Link] is present in the target
directory. This approach is chosen because Quarkus incorporates
customized logic to accelerate the boot process, even in Java mode.
Upon running the application, you can expect a quicker start-up, which
closely aligns with Micronaut's performance. Test the API using commands
like:
// employing the default name parameter
$ curl [Link]
{«details»:»Hello, Java!»}
Or
// using a specific value for the name parameter
$ curl [Link]
{"details":"Hello, Microservs!"}

Quarkus supports the generation of native executables through GraalVM

Native Image, aligning with its focus on cloud environments and a
preference for smaller binary sizes. Much like Micronaut, Quarkus provides
a comprehensive set of tools right from the start, eliminating the need for
any build configuration updates to begin working with native executables.
Similar to the previous examples, you will need to ensure your current JDK
is pointed to a GraalVM distribution, and that the native-image executable
is found in your path. Once these prerequisites are met, packaging the
application as a native executable is straightforward. Just run the command
./mvnw -Pnative package, which activates the native profile and instructs
Quarkus build tools to generate the native executable.
After a short wait, the build should have produced an executable named
app-1.0.0-SNAPSHOT-runner within the target directory. Executing this
executable shows that the application boasts a quicker start-up time,
establishing Quarkus as the framework with the quickest start-up and the
smallest executable size when compared to previous candidate frameworks.
So far, we have explored each framework's distinctive features and
integration aspects, covering foundational elements and build tool
compatibility. As a helpful reminder, it is essential to consider specific
features and aspects relevant to your development needs when choosing a
framework. It is recommended to create a matrix for each critical feature or
aspect and evaluate each candidate against those criteria. This systematic
approach helps ensure an informed decision that aligns with the
development requirements.

Serverless
So far in this chapter, we have seen the journey from Monolithic
architecture to Microservices. Each of them brought its own set of
challenges. While Microservices promised independent deployments and
scalability, managing the orchestration of these services introduced
complexities. Enter the realm of serverless computing and Function as a
Service (FaaS), where the focus shifts from infrastructure management to
code execution.
Traditional Monolithic applications often meant bundling multiple
components tightly, making updates cumbersome and failures potentially
catastrophic. Microservices alleviated some of these concerns but
introduced new challenges, such as coordinating multiple instances, dealing
with network latency, and managing distributed systems.
Serverless architecture emerges as a solution, allowing developers to
concentrate solely on business logic. In this paradigm, the serverless
provider handles infrastructure, scaling, monitoring, and other operational
aspects, liberating developers from these concerns.
In the process of breaking down a component into smaller parts, it is crucial
to consider: What's the smallest piece of reusable code within this
component? If you are thinking of a Java class with a few methods and
some injected collaborators/services, you are on the right track, but there is
a smaller unit, a single method. Imagine a Microservice represented by a
class that follows these steps:
1. Takes input arguments and transforms them into a usable format for
the next step.
2. Executes the required behavior, like querying a database, indexing,
or logging.
3. Converts the processed data into a designated output format.
Although each step can have its own methods, some may be reusable as-is
or parameterized. One common approach is to introduce a common super
type among Microservices, fostering strong type dependencies. While
suitable for some scenarios, it poses challenges in updating common code
promptly without disrupting ongoing operations. In such cases, an
alternative is needed.
Consider a scenario where common code is presented as a set of
independently invokable methods, with inputs and outputs structured to
form a pipeline of data transformations. This concept aligns with what is
now recognized as functions, and services like FaaS are commonly
discussed in the realm of serverless providers.
In short, FaaS is a sophisticated way of expressing that you construct
applications using the tiniest deployable units, leaving the complex details
of infrastructure management to the service provider. In the upcoming
sections, we will create and deploy a straightforward function to the cloud.

Setting up
In the present-day context, each major cloud provider offers a FaaS
solution, complete with additional features that seamlessly integrate with
various tools for tasks such as monitoring, logging, and disaster recovery.
Simply, choose the one that aligns with your specific requirements. For the
purposes of this chapter, we will opt for AWS Lambda, considering it
pioneered the FaaS concept. Additionally, we will select Quarkus as our
implementation framework, given its current status as the framework with
the smallest deployment size. It is important to note that the configurations
presented here may require adjustments or could potentially be outdated.
So, always review the most recent versions of the necessary tools for
building and running the code. We will be using the latest version of
Quarkus for this endeavor. Please follow the below steps for end-to-end
execution :
1. Prepare the environment: To establish a function with Quarkus
and AWS Lambda, ensure you have an AWS account, install the
AWS CLI on your system, and if you wish to conduct local tests,
install the AWS Serverless Application Model (SAM) CLI.
2. Project Initialization: Once you have handled that, the next phase
is project initiation. In this case, we will be opting for Quarkus as
done previously, except considering that the function project needs a
 different setup. Hence, it is advisable to make the shift to using a
Maven archetype:
mvn archetype:generate \
-DarchetypeGroupId=[Link] \
-DarchetypeArtifactId=quarkus-amazon-lambda-archetype \
-DarchetypeVersion=[Link]
Executing the preceding command in interactive mode prompts
several inquiries, including the Group, Artifact, and Version
(GAV) coordinates for the project, along with the base package. For
this demonstration, let us opt for the following:
• groupId: [Link]
• artifactId: app
• version: 1.0-SNAPSHOT (this is the default)
• package: [Link] (same as groupId)
This leads to a project structure suitable for constructing, testing,
and deploying a Quarkus project as a function deployable on AWS
Lambda.
3. Verify the generated project structure: The archetype generates
build files for Maven and Gradle, although we currently only require
Maven. Additionally, it also generates three function classes
([Link], [Link] and
[Link]), but we will focus on utilizing just one
([Link]). Our objective is to have a file
structure similar to the following:
├── [Link]
├── [Link]
└── src
├── main
│ ├── java
│ │ └
 │ │ └── com
│ │ └── myproject
│ │ └── app
│ │ ├── [Link]
│ │ ├── [Link]
│ │ ├── [Link]
│ │ └── [Link]
│ └── resources
│ └── [Link]
└── test
├── java
│ └── com
│ └── myproject
│ └── app
│ └── [Link]
└── resources
└── [Link]
4. Understand Function details: The essence of the function revolves
around capturing inputs of the CustomInputObject type, handling
their processing through the CustomProcessingService type, and
subsequently converting the results into another type
(CustomOutputObject). The CustomGreetingLambda type
serves as the orchestrator, bringing all these components together.
First, let us examine both the input and output types. After all, they
are straightforward types focused solely on containing data, devoid
of any logic. The following is the code for CustomInputObject
class:
package [Link];
public class CustomInputObject {
private String personName;
private String personalizedGreeting;
public String fetchPersonName() {
return personName;
}
public void assignPersonName(String personName) {
[Link] = personName;
}
public String getPersonalizedGreeting() {
return personalizedGreeting;
}
public void setPersonalizedGreeting(String personalizedGreetin
g) {
[Link] = personalizedGreeting;
}
}
The lambda requires two input values: a greeting and a name. We
will see how these values get transformed by the processing service.
For now, let us look at the following CustomOutputObject class:
package [Link];
public class CustomOutputObject {
private String details;
private String requestId;
public String getOperationResult() {
 return details;
}
public void setOperationResult(String details) {
[Link] = details;
}
public String getUniqueRequestId() {
return requestId;
}
public void setUniqueRequestId(String requestId) {
[Link] = requestId;
}
}
The CustomOutputObject class serves as a structured container for
holding the result of an operation (result) and a unique identifier
associated with the request (requestId).
Now, let us move on to the processing service; this class converts
inputs into outputs. In our scenario, it combines the value of both
inputs into a single string, as illustrated as follows:
package [Link];
import [Link];
@ApplicationScoped
public class CustomProcessingService {
public CustomOutputObject process(CustomInputObject input)
{
CustomOutputObject output = new CustomOutputObject();
 [Link]([Link]() +
" " + [Link]());
return output;
}
}
The final step is to have a look at CustomGreetingLambda, the
class responsible for constructing the function itself. This class
needs to implement a well-known interface provided by Quarkus.
The necessary dependency for this interface should already be set up
in the [Link] file generated by the archetype. The interface is
designed with input and output types, and fortunately, we already
have those. Each lambda must have a unique name and can access
its runtime context, as demonstrated as follows:
package [Link];
import [Link];
import [Link];
import [Link];
import [Link];
@Named("greet")
public class CustomGreetingLambda
implements RequestHandler<CustomInputObject, CustomO
utputObject> {
@Inject
CustomProcessingService service;
@Override
public CustomOutputObject handleRequest(CustomInputObject
input, Context context) {
 CustomOutputObject output = [Link](input);
[Link]([Link]());
return output;
}
}
This lambda function (CustomGreetingLambda) is designed to
handle greetings. It takes a CustomInputObject, processes it using
a CustomProcessingService, associates a unique request ID
obtained from the Lambda context, and returns the processed result
as a CustomOutputObject. The function is suitable for deployment
as an AWS Lambda function that can be triggered by events.
Now, everything comes together seamlessly. The lambda defines
input and output types and calls the data processing service. In this
example, dependency injection is used for demonstration purposes,
but for a more streamlined code, you could consolidate the
behaviour of CustomProcessingService into
CustomGreetingLambda. To validate the code, you can run local
tests using mvn test, or mvn verify can be used as it also packages
the function.
Please be aware that when the function is packaged, extra files are
placed in the target directory. One of these files is a script named
[Link], which utilizes the AWS CLI tool to perform functions
such as creating, updating, and deleting the function at the specified
target destination associated with your AWS account. To facilitate
these operations, certain additional files are necessary:
• [Link]: This deployment file holds the binary bits of the
function.
• [Link]: This YAML file is used for local testing with
the AWS SAM CLI in Java mode.
• [Link]: This YAML file is utilized for local testing
with the AWS SAM CLI in native mode.
5. Configure execution role: The subsequent step involves
configuring an execution role, and it is recommended to consult the
AWS Lambda Developer Guide for the most up-to-date procedure.
The guide provides instructions on configuring the AWS CLI (if not
already done) and creating an execution role. This role needs to be
included as an environment variable in your running shell. Refer the
following example:
LAMBDA_ROLE_ARN="arn:aws:iam::1236789450:role/lamb
da-rol"
In this example, 1236789450 represents your AWS account ID, and
lambda-rol is the name you choose for the role. Now, we can move
on to executing the function, which we can perform via two modes
(Java mode and native mode) and two execution environments
(local and production). First, we will explore Java mode for both
environments and then proceed to native mode.
6. Testing on local machine: To run the function locally, you need a
Docker daemon, which is commonly available in a developer’s
toolbox. Additionally, the AWS SAM CLI is required to drive the
execution. Remember the extra files in the target directory? We will
utilize the [Link] file along with another file, [Link],
created by the archetype during project bootstrapping. This file,
found at the root of the directory, should have contents similar to the
following:
{
"personName": "David",
"personalizedGreeting": "hello"
}
This file specifies values for the inputs accepted by the function.
Since the function is already packaged, we just need to invoke it, as
demonstrated as follows:
$ sam local invoke --template target/[Link] --event paylo
[Link]
Invoking [Link]::handleRequest
(java17)
Decompressing /work/app/target/[Link]
Skip pulling image and use local one:
amazon/aws-sam-cli-emulation-image-java17:rapid-1.24.1.
Mounting /private/var/folders/p_/4h16jn992hr0zs1ckrn9nk0m0
000ho/T/tmpnrkk0c7 as
/var/task:ro,delegated inside runtime container
START RequestId: 0d6ebf4a-fdc3-42e1-bf36-56a470324af1 Vers
ion: $LATEST
__ ____ __ _____ ___ __ ____ ______
--/ __ \/ / / / _ | / _ \/ //_/ / / / __/
-/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
[[Link]] (main) Quarkus [Link] on JVM started in 2.67
0s.
[[Link]] (main) Profile prod activated.
[[Link]] (main) Installed features: [amazon-lambda, cdi]
END RequestId: 0d6ebf4a-fdc3-42e1-bf36-56a470324af1
REPORT RequestId: 0d6ebf4a-fdc3-42e1-bf36-56a470324af1 In
it Duration: 1.70 ms
Duration: 3261.01 ms Billed Duration: 3200 ms
Memory Size: 256 MB Max Memory Used: 256 MB
{"details":"hello David","requestId":"0d6ebf4a-fdc3-42e1-bf36
-56a470324af1 "}
Executing this command will pull a Docker image suitable for
executing the
function. Observe the provided values, as they might vary based on
your configuration. In my local environment, running this function
 would take approximately 3200 ms and consume 256 MB of
memory. This information can offer insight into the potential billing
when operating your system as a set of functions. However, it is
crucial to note that the local environment differs from the production
environment.
7. Production deployment: To deploy the function to the actual
environment, we will utilize the [Link] script and execute the
following commands:
$ sh target/[Link] create
$ sh target/[Link] invoke
Invoking function...
++ aws lambda invoke [Link] --cli-binary-format raw-in-b
ase64-out
++ --function-name QuarkusLambda --payload [Link]
on
++ --log-type Tail --query LogResult
++ --output text base64 --decode
START RequestId: bf8d17ad-1e64-4ace-a54b-93b8a09341c9 Ver
sion: $LATEST
END RequestId: bf8d17ad-1e64-4ace-a54b-93b8a09341c9
REPORT RequestId: bf8d17ad-1e64-4ace-a54b-93b8a09341c9
Duration: 271.47 ms
Billed Duration: 273 ms Memory Size: 256 MB
Max Memory Used: 123 MB Init Duration: 1631.69 ms
{"details":"hello David","requestId":"bf8d17ad-1e64-4ace-a54
b-93b8a09341c9"}
As evident, the billed duration and memory usage have reduced,
which is advantageous for cost considerations. However, the init
duration has increased, potentially leading to delayed responses and
an overall rise in the total execution time across the system.
8. Switch to Native mode: To observe the impact of transitioning from
Java mode to native mode, let us assess how these metrics change.
To remind you, By default Quarkus allows you to package projects
as native executables by default. Nevertheless, Lambda requires
Linux executables. If you are running on a non-Linux environment,
adjustments to the packaging command are required. Here is the
necessary steps:
To build the native executable on a Linux system, use the following
Maven command:
$ mvn -Pnative package
For non-Linux environments, run the subsequent Maven command
with added parameters to manage containerization:
$ mvn package -Pnative -[Link]-build=true \
-[Link]-runtime=docker
The second command triggers the build inside a Docker container,
placing the resulting executable at the specified location on your
system. Conversely, the first command runs the build in its original
form. Now that the native executable is available, we can run the
updated function in both local and production environments.
9. Testing on Local machine ( Native Mode): Let us begin with
running this on the local environment:
$ sam local invoke --template target/[Link] --event pay
[Link]
Invoking [Link] (provided)
Decompressing /work/app/target/[Link]
Skip pulling image and use local one:
amazon/aws-sam-cli-emulation-image-provided:rapid-1.24.1.
Mounting /private/var/folders/p_/4h16jn992hr0zs1ckrn9nk0m0
000ho/T/tmp1nrjj06 as
/var/task:ro,delegated inside runtime container
 START RequestId: 26521b6d-351c-24f7-71b3-533cd5ec7de4 Ver
sion: $LATEST
__ ____ __ _____ ___ __ ____ ______
--/ __ \/ / / / _ | / _ \/ //_/ / / / __/
-/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
[[Link]] (main) quarkus-lambda 2.6.1 native
(powered by Quarkus [Link]) started in 0.114s.
[[Link]] (main) Profile prod activated.
[[Link]] (main) Installed features: [amazon-lambda, cdi]
END RequestId: 4b516d7c-164b-54e6-29e4-466ed7df8fd4
REPORT RequestId: 4b516d7c-164b-54e6-29e4-466ed7df8fd4 I
nit Duration: 0.13 ms
Duration: 216.76 ms Billed Duration: 312 ms Memory Size: 128
MB
Max Memory Used: 128 MB
{"details":"hello David","requestId":"4b516d7c-164b-54e6-29e
4-466ed7df8fd4"}
The billed duration reduced significantly, dropping from 3200 ms to
just 312 ms, and the utilized memory was cut in half. This seems
encouraging in comparison to its Java equivalent. Can we expect
even better performance in a production environment? Let us check.
10. Production deployment (Native Mode): We will be deploying and
invoking native version using the [Link] script :
$ sh target/[Link] native create
$ sh target/[Link] native invoke
Invoking function
++ aws lambda invoke [Link] --cli-binary-format raw-in-b
ase64-out
 ++ --function-name QuarkusLambdaNative
++ --payload [Link] --log-type Tail --query LogResul
t --output text
++ base64 --decode
START RequestId: 14565bd3-3220-405c-bfa0-76ab52e7c8b5 Ver
sion: $LATEST
END RequestId: 14565bd3-3220-405c-bfa0-76ab52e7c8b5
REPORT RequestId: 14565bd3-3220-405c-bfa0-76ab52e7c8b5
Duration: 2.55 ms
Billed Duration: 184 ms Memory Size: 256 MB Max Memory Us
ed: 54 MB
Init Duration: 163.71 ms
{"details":"hello David","requestId":"14565bd3-3220-405c-bfa
0-76ab52e7c8b5"}
The billed duration again reduced significantly, dropping from 273
ms ( Java production mode) \ to 184 ms, and the memory usage is
now less than half of its previous state; but the standout
improvement is in the initialization time, which now takes roughly
10% of the time it took in Java mode. Running your function in
native mode results in a faster start-up and better numbers across the
board. Note that all these numbers will be different for you based on
various configurations.
The choice of the best combination of options is in your hands. Staying in
Java mode might be sufficient for production in some cases, while going
entirely native could provide a competitive advantage. Regardless of your
decision, accurate measurements are essential—avoid relying on
guesswork!

Conclusion
Monolithic architecture is a single, tightly integrated system, while
Microservices break it into smaller, independently deployable services. The
shift from Monolithic to Microservices is driven by the need for greater
flexibility, scalability, and agility.
This chapter outlined key differences between the two, highlighting
challenges in Monolithic applications and when Microservices are the better
choice. It also covered the planning required for a smooth transition,
offering step-by-step guidance, best practices, and the role of DevOps in
supporting Microservices. The chapter reviewed Microservices frameworks
like Spring Boot, Micronaut, and Quarkus, and concluded with practical
insights on serverless architecture using AWS Lambda and Quarkus. In
essence, the chapter serves as a comprehensive guide to the entire spectrum
of transitioning to Microservices, encompassing planning, DevOps
integration, Microservices frameworks, and practical implementation of
serverless computing.
In the next chapter, we will look closely at source code management.

Join our book’s Discord space

Join the book’s Discord Workspace for Latest updates, Offers, Tech
happenings around the world, New Release and Sessions with the Authors:
[Link]
 CHAPTER 3
Manage Your Source Code

Introduction
Every software product begins with an idea, which is implemented in
source code. These source code files make our applications work, so we
must treat them with care. To ensure that we have a single system of truth,
we keep them safe and track a history of changes associated with it. We also
want to allow for smooth collaboration between multiple team members.
Typically, this starts with a source control management system that contains
all the source code that gets compiled and built into the production
deployment. By tracing a production deployment back to a specific revision
in source control, you can do root cause analysis of bugs, security holes,
and performance issues. A good source control management makes us
understand what is being deployed on production and creates an effective
DevOps process and pipelines.

Structure
In this chapter, we will discuss the following topics:
• Source control management
• Three generations of source control management
 • Code versioning
• Choosing your source control tool
• Git components
• Making your first pull request
• Git tools
• Git workflows
• Best practices for source control management

Objectives
Upon completing this chapter, you will gain a comprehensive understanding
of source control management (SCM) and Git, a widely used distributed
version control system. The chapter covers three generations of SCM,
explores the concept of code versioning, and guides you through the
process of choosing the most suitable source control management system
for their projects.

Source control management

Source control management is the practice of tracking changes to source
code. Keeping a history of all the changes made to a codebase helps
developers and testers to ensure that they are always working with correct
and up-to-date pieces of code. This also helps resolve code conflicts that
occur when multiple sources merge the code.
When multiple developers work on a shared code base, source control
management is essential, especially when each developer is working on
their respective feature, and unknowingly, they could make conflicting code
changes, or one developer’s work could overwrite the other developer’s
changes.
Before the adoption of source control management, each time a developer
started working on a code file, they would have to tell others so that no one
else worked on it simultaneously. However, that was not fault-prone. Any
miscommunication could result in code conflicts and code rework. Another
drawback of this manual process was that tracking the change history was a
big headache, especially, when any change resulted in a bug.
Source control management solves several key problems related to code in
the software delivery lifecycle and provides immense benefits:
• Collaboration: Without proper source control management, team
members of large teams working on a single codebase would
frequently get blocked by one another. This impacts productivity.
• Protection: It helps teams manage the source control and prevents it
from being deleted, accidentally lost, or compromised by human
error.
• Versioning: Versioning of code is useful in identifying what is
being deployed or delivered to customers.
• History: Preserving the older code files in chronological order helps
in restoring the system with previous versions if any issues occurred
or performance degraded with current changes.
• Attribution: Maintaining the record of the author of the changes in
a particular file allows one to identify who does the changes,
evaluate the expertise and risks when making changes.
• Dependencies: With source control management repositories, it is
easier to manage project metadata and other dependencies.
• Quality: It allows an easy review process of changes, which
provides better and quick feedback. This increases the overall
quality of the product.
Source control management tracks each developer’s changes, warn and
highlight conflicts so they can address them before they are merged into the
source control and prevents code overwriting. Since source control
management plays such a critical role in software development, it is
important to understand how it works and select a system that best meets
the needs of your organization and the desired DevOps workflow.
Three generations of source control management
Evolution in SCM has seen distinct generations, each addressing challenges
and enhancing collaboration in software development. From the initial era
of local file-centric control to the centralized repositories of the second
generation, and finally, the distributed versatility of the third generation,
these advancements have significantly influenced developer workflows and
productivity. Let us discuss the characteristics and tools defining each
generation of SCM.

First generation: Local SCM software

The first generation SCM was intended to track changes for individual files.
You have to check out (lock) files before you edit them. Then, you check in
(release the lock) the files for other members to edit. This eliminated the
possibility of two developers making changes at the same time, making
conflicting changes. While using this SCM, you had to wait for the other
person to finish his/her changes and release the lock; this resulted in
affecting the overall productivity of the team.
A few tools of first-generation source control management are:
• Revision Control System (RCS)1
• Source Code Control System (SCCS)2

Second generation: Centralized SCM tools

With time, a substantial improvement was made in second generation SCM
tools. The second generation introduced centralized repositories containing
the official versions of the code. A centralized source control management
system allowed multiple users to make changes in the code at the same
time. Later, all conflicts were analyzed and conflict changes were given
back to developers to resolve. Once resolved, the code was committed back
to the same central repository. This not only increased the developer
productivity but also allowed developers to work individually in isolation
and test the large features separately, which could later be merged into an
integrated codebase.
A few second generation source control management tools include:
• Perforce Helix Core
• Concurrent Versions System (CVS)
• Apache Subversion (SVN)
• Microsoft Visual SourceSafe (VSS)

Third generation: Distributed SCM systems

The third generation of source control management system comprises the
distributed VCS, or DVCS. In distributed VCS, every developer has a copy
of the entire repository and the full history stored locally. You check out a
copy of the repository, make changes, and check in back. If you want to
integrate those changes with other developers, you sync your entire
repository in a peer-to-peer fashion. With DVCS SCM, you have a local
copy of the repository, checking code in and out, merging, and managing
branches can all be done without a network connection. All SCM
operations are local to the machine and not affected by network speed or
server load, hence, they are faster. The initial sync includes copying the
entire repository history, which can be much slower. Since everyone has a
full copy of the repository and all history, very large projects may require an
extended disk requirement.
The third-generation source control management tools have been listed as
follows:
• Git
• Mercurial
• BitKeeper
• Darcs Advanced Revision Control System (Darcs)
• Fossil
 • TFS

Code versioning
Code versioning is the numbering of different releases of a particular
software product for both internal use and release. It allows the developers
to know when changes have been made and track the changes. The changes
may include new functions, features, or bug fixes. It also enables end users
to recognize the new releases and updated versions.
Versioning practice makes it easier for developers to keep track of code
evolution. During development, teams apply internal versioning numbers
that can be incremented several times within a day. In contrast, the publicly
released version usually does not change very often.

Semantic versioning
There are many ways of versioning software products and artifacts. One of
the most popular schemes is semantic versioning. Semantic Versioning is a
3-component number in the format of [Link]. While
doing the versioning increment:
• MAJOR version when you make incompatible API changes
• MINOR version when you add functionality in a backwards-
compatible manner
• PATCH version when you make backwards-compatible bug fixes
Additional labels for pre-release and build metadata are available as
extensions to the [Link] format. Please check
[Link] for more details on semantic versioning.

Best practices for code versioning

The following are some of the best practices for code versioning:
• Start new projects at version 0.1.0 and increment the minor version
for each subsequent release.
 Start versioning at 1.0.0 if your software is already used in
• production. If you have a stable API on which users have come to
depend, you should be 1.0.0. If you are worrying about backwards-
compatibility, you should already be 1.0.0.
• The initial development phase is represented by MAJOR Version 0.
You can make as many breaking changes as you want before v1.0.0.
• If you are adding new features without breaking the existing API or
functionality, increment the MINOR number.
• If you are fixing bugs, increment the PATCH number.
You can get more information on semantic versioning visit
[Link]

Choosing your source control

As shown in the previous sections, a modern DVCS provides the best
capabilities for local and remote development of any size team.
Among all the commonly used source control management tools, Git has
become the winner in adoption. The following trend analysis in Figure 3.1
shows this clearly:

Figure 3.1: Interest in SCM tools from 2004 through 20233

Figure 3.2 shows the top five version control technologies with their
percentage market share:
 Figure 3.2: Top five version control technologies in 20234

Git is popular in the open-source community, which means a broad base of

support exists for its usage. However, sometimes, convincing your manager
or peers to adopt new technologies is difficult if your company has a huge
investment in a legacy source control tool.
Here are some reasons why you should upgrade to Git:
• Faster response time: Its response time is faster, which saves a lot
of time. You can use your time for something more useful than
waiting for your version control system to get back to you.
• Reliable: Git is written like a filesystem, including a proper
filesystem check tool (git fsck) and checksums to ensure data
reliability. So, you can have your data backed up to multiple
external repositories.
• Performance: Git is extremely performant. It was built from the
ground up to support Linux development with huge codebases and
thousands of developers. Git continues to be actively developed by a
large open-source community. Git is extremely performant on
MacOS and Windows as well.
 • Good tooling support: There are a lot of front ends for Git, and it
has support in every major IDE (IntelliJ IDEA, Microsoft Visual
Studio Code, Eclipse, Apache NetBeans, etc.). Hence, it is very
unlikely that any development platform will not fully support it. Git
has excellent integrations with IDEs, issue trackers, messaging
platforms, continuous integration servers, security scanners, code
review tools, dependency management, and cloud platforms.
• Migration tools support: There are migration tools for transition
from other version-control systems to Git, such as git-svn that
supports bidirectional changes from Subversion to Git, or the Team
Foundation Version Control (TFVC) repository import tool for
Git.
When you upgrade to Git, you can have lots of additional capabilities and
integrations to take advantage of. Getting started with Git is simple. You
have to download it for your development machine and create a local
repository. However, the real power of Git comes when collaboration take
place with the rest of your team, and it is most convenient if you have a
central repository to push changes to. Several companies offer commercial
Git repos that you can self-host or run on their cloud platform. These
include AWS CodeCommit, Azure DevOps, GitLab, GitHub, SourceForge,
Bitbucket, and others.

Git components
Before we dive into the world of Git commands, let us take a step back and
visualize an overview of the components that make up the Git ecosystem.
Figure 3.3 shows how the components work together:
 Figure 3.3: Git components

Git GUI tools act as a front-end for the Git command line, and some tools
have extensions that integrate with popular Git hosting platforms. The Git
client tools mostly work on the local copy of your repository.
When working with Git, a typical setup includes a Git server and clients.
The Git server and clients work as follows:

Git server
A Git server enables you to collaborate more easily because it ensures the
availability of a central and reliable source of truth for the repositories you
will be working on. A Git server is also where your remote Git repositories
are stored; as common practice goes, the repository has the most up-to-date
and stable source of your projects. You have the option to install and
configure your own Git server, or you can opt to host your Git repositories
on reliable third-party hosting sites such as GitHub, GitLab, and Bitbucket.

Git clients
Git clients interact with your local repositories, and you can interact with
Git clients via the Git command-line or the Git GUI tools. When you install
and configure a Git client, you can access the remote repositories, work on
a local copy of the repository, and push changes back to the Git server.

Making your first pull request

Now, we will run through a simple exercise to create your first pull request
to the official repository on GitHub. This will give you insight into how
version control works. This exercise does not require installing any
software or using the command line, so it should be easy and
straightforward to accomplish. Once you finish this exercise, you will
understand the basic concepts of distributed version control, which we will
go into more detail about, later in this chapter.
1. For this exercise, you need to be logged in so you can create a pull
request from the web user interface. If you do not already have a
GitHub account, signing up and getting started is easy and free.
2. Create a repository named DevOps and navigate to the DevOps
repository.
3. The Source-Code-Management repository GitHub page is shown in
Figure 3.4. By default, the GitHub UI shows the root files and the
contents of a special file called [Link]. We are going to add
a text file here.
4. Since we just have the read only access to this repository, we are
going to create a personal clone of the repository, known as a fork,
that we can edit to make and propose the changes. Once you are
logged in to GitHub, you can start this process by clicking the Fork
button highlighted in the upper-right corner.

Figure 3.4: The GitHub repository containing this book’s samples code
5. The new fork will get created under your personal account at
GitHub. Once the fork is created, complete the below steps to open
the web-based text editor:
a. Click on the new file.
b. Write the name of file and write content on web-based text
editor shown in Figure 3.5:

Figure 3.5: The GitHub web-based text editor for adding content

c. When satisfied with your change, scroll down to the code

commit section shown in Figure 3.6. Enter a description for the
change. Then go ahead and click the Commit changes button.
This will commit to the main branch, which is the default.
However, if you are working in a shared repository, you would
commit your pull request to a feature branch that you can
integrate separately.

Figure 3.6: Commit changes to a git repository you have write access to

6. After you add a file to your forked repository, you can submit this as
a pull request for the original project. This will notify the project
 maintainers that a proposed change is waiting for review, and upon
verification, they can integrate it into the original project.
7. To merge this change, go to the Pull requests tab in the GitHub user
interface. You will get a button to create a New pull request that
will present you with a choice of the “base” and “head” repository
to be merged, as shown in Figure 3.7. Now, click the Create pull
request button, and a new pull request against the original
repository will be submitted for review.

Figure 3.7: Creating a pull request from the forked repository

8. This completes your submission of a pull request! Now, it is up to

the original repository owners to review and accept/reject the pull
request. Figure 3.8 shows what will be presented to the repository
owners. Once the repository owners accept your pull request, your
file will be added to the main repository.
 Figure 3.8: The repository owner web interface for merging the pull request

This workflow is an example of the using fork and pull request model for
doing the project integration. We will talk a bit more about other workflows
in the Git workflows section.

Git tools
In the previous section, we went through an entire web-based workflow for
Git using the GitHub UI. However, majority of the developers spend their
time in one of the client-based user interfaces to Git. The available client
interfaces can be broadly split into the following categories:
• Command line: An official Git command-line client can easily be
added to your machine.
• GUI clients: The official Git distribution comes with a couple of
open-source tools that can be used to more easily browse your
revision history or to structure a commit. Also, there are several
third-party free and open-source Git tools in the market.
• Git IDE plug-ins: Often, you need your favorite IDE to work with
your distributed source control system. Many major IDEs offer a
well-supported plug-in for git.

Git command-line basics

Git’s command-line interface is simple to use. It is designed to give you full
control of your repository. We will be focusing on the commands that are
important for your day-to-day work. we will simplify some advanced
commands as well.
As a starting point, just type git version or git --version to determine
whether your machine has already been preloaded with Git. You should see
output similar to the following:
$ git --version
git version [Link].1
If you do not have Git installed on your machine, please follow these
instructions to install it in an easy way:
• Linux distributions:
○ Debian/Ubuntu: sudo apt install git-all
○ Fedora systems: sudo dnf install git-all
• macOS:
○ Homebrew:
$ brew install git
○ MacPorts:
# Requires MacPorts to be installed prior to installing Git
$ sudo port install git
○ Another easy option is to install GitHub Desktop, which
installs and configures the command-line tools.
• Windows: On Windows, you have the option to install Git from a
standalone installer, from a portable installer, or by using the
Windows Package Manager, winget, or the community-maintained
package manager, chocolatey:
○ Install GitHub Desktop, which installs the command-line tools
as well.
○ Winget:
# Execute from Powershell or Command Prompt
PS C:\>winget install --id [Link] -e --source winget
○ Chocolatey:
# Execute from Powershell or Command Prompt
PS C:\>choco install [Link]
Install Git according to your operating system platform before continuing
with the next section.
Upon installation, type git. Git will then list its options and the most
common subcommands:
$ git
Usage:
git [-v | --version] [-h | --help] [-C <path>] [-c <name>=<value>]
[--exec-path[=<path>]] [--html-path] [--man-path] [--info-pat
h]
[-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bar
e]
[--git-dir=<path>] [--work-tree=<path>] [--namespace=<na
me>]
[--super-prefix=<path>] [--config-env=<name>=<envvar>]
<command> [<args>]
These are common Git commands used in various situations:
• Start a working area (see also: git help tutorial)
○ clone: A repository into a new directory
○ init: Create an empty Git repository or reinitialize an existing
one
• Work on the current change (see also: git help everyday)
○ add: File contents to the index
○ mv: Move or rename a file, a directory, or a symlink
○ restore: working tree files
○ rm: Remove files from the working tree and from the index
• Examine the history and state (see also: git help revisions)
○ bisect: Use binary search to find the commit that introduced a
bug
 ○ diff: Show changes between commits, commit and working
tree, etc
○ grep: Print lines matching a pattern
○ log: Show commit logs
○ show: various types of objects
○ status: Show the working tree status
• Grow, mark and tweak your common history
○ branch: List, create, or delete branches
○ commit: Record changes to the repository
○ merge: Join two or more development histories together
○ rebase: Reapply commits on top of another base tip
○ reset: current HEAD to the specified state
○ switch: branches
○ tag: Create, list, delete or verify a tag object signed with GPG
• Collaborate (see also: git help workflows)
○ fetch: Download objects and refs from another repository
○ pull: Fetch from and integrate with another repository or a
local branch
○ push: Update remote refs along with associated objects
• git help -a and git help -g lists subcommands and some concept
guides.
See git help <command> or git help <concept> to read about a
specific subcommand or concept.
See git help git for an overview of the system.

Tips:
1. To get a complete list of git subcommands, type git help --all.
 2. As you can see from the usage hint, a small handful of options
apply to git. Most options, shown as [ARGS] in the hint, apply
to specific subcommands. For example, the option --version
affects the git command and produces a version number:
$ git –version
git version [Link].1
In contrast, --amend is an example of an option specific to the
git subcommand commit:
$ git commit –amend
Some invocations require both forms of options:
$ git --git-dir=[Link] repack -d
3. For convenience, documentation for each git subcommand is
available using git help
subcommand, git --help subcommand, git subcommand --help,
or man git-subcommand.

The complete Git documentation5 is available online.

Historically, Git was provided as a suite of many simple, distinct,
standalone commands developed according to the Unix philosophy: build
small, interoperable tools. Each command exhibited a hyphenated name,
such as git-commit and git-log. However, modern Git installations no
longer support the hyphenated command forms and instead use a single git
executable with a subcommand.
The git commands understand both “short” and “long” options. For
example, the git commit command treats the following examples
equivalently:
$ git commit -m “My First commit”
$ git commit --message=”My First Commit”
The short form, -m, uses one hyphen, whereas the long form, --message,
uses two.

Tip: Using the -m option multiple times you can create a commit
summary and detailed message for the summary:
$ git commit -m “Summary” -m “Detail of Summary”

To start with, it is helpful to understand the basic Git commands. Figure 3.9
shows a typical repository hierarchy with one central(remote) repository
who have been cloned locally. When a developer clones the repository, they
get two things on their local machine: a full copy of the entire repository
(with all its history) and a working directory where they can make changes
to the files. The working directory is where the developer writes and edits
code. As the developer makes changes, they can stage these changes,
commit them to their local repository, and later push them to the central
(remote) repository when they are ready to share their work with others.

Figure 3.9: A typical git repositories hierarchy with remote repository and local repository

Preparing to work with Git from command line

Whether you are creating a new repository or working with an existing
repository, there are basic prerequisite configurations that you need to do
after installing Git on your local machine. At a bare minimum, Git requires
your name and email address before you make your first commit in your
repository. The identity you supply then shows as the commit author, baked
in with other snapshot metadata. You can save your identity in a
configuration file using the git config command:
$ git config [Link] "Neeraj kumar"
$ git config [Link] "neerajdevopsdemo@[Link]"
If you decide not to include your identity in a configuration file, you will
have to specify your identity for every git commit subcommand by
appending the argument --author at the end of the command:
$ git commit -m "First commit" --author="Neeraj Kumar <neer
ajdevopsdemo@[Link] >"
Keep in mind that this is the hard way, and it can quickly become
monotonous.
You can also specify your identity by supplying your name and email
address to the GIT_AUTHOR_NAME and GIT_AUTHOR_EMAIL
environment variables, respectively. If you set, these variables will override
all configuration settings.

Working with a local repository

Now that you have configured your identity, you are ready to start working
with a repository. We will be performing the following tasks:
• Creating a new empty repository on your local development
machine.
• We will be making that repository a Git repository.

Creating an initial repository

We will model a typical situation by creating a repository. Let us assume
you are starting from scratch and will add content for your project in the
local directory ~/First-Demo, which you place in a Git repository.
Type the following commands to create the directory, and place some basic
content in a file called [Link]:
 $ mkdir ~/First-Demo
$ cd First-Demo/
$echo ‘My First commit’ > [Link]
To convert ~/ First-Demo into a Git repository, run git init. Here, we
provide the option -b followed by a default branch named main:
$ git init -b main
Initialized empty Git repository in /First-Demo/.git/
Another way of doing this is to initialize an empty Git repository first and
then add files to it. You can do so by running the following commands:
$ git init -b main ~/First-Demo
Initialized empty Git repository in ../First-Demo /.git/.
$ cd ~/First-Demo
$ echo 'My First commit' > [Link]

Tips:
1. Either you initialize a completely empty directory or an
existing directory full of files, the process of converting the
directory into a Git repository is the same.
2. When you create a new repository with git init, by default Git
creates a branch called master. From Git version 2.28
onwards, you can set a different name for the initial branch.
To set main as the default branch name do:
$ git config --global [Link] main

Now, let us look at some basic and advanced commands that are used to
manage your repository and collaborate in Git:
• Branches: Branches are an important part of working with Git. Any
commits you make will be made on the branch you are currently
“checked out” to. Use git status to see which branch that is.
$ git branch [branch-name]
 Creates a new branch
git checkout [branch-name]
navigate between the branches created by git branch command and
updates the
working directory
git merge [branch]
Joins changes from the named commits into the current branch. If
the merged history is already a descendant of the current branch, a
“fast-forward” is used to combine the history sequentially.
Otherwise, a merge is created that combines the history; the user is
prompted to resolve any conflicts. This command is also used by git
pull to integrate changes from the remote repository.
git branch -d [branch-name]
Deletes the specified branch
• Create repositories and manage repositories: When starting out
with a new repository, you only need to do it once; either locally,
then push to GitHub, or by cloning an existing repository.
$ git init
Turn an existing directory into a git repository.
$ git clone [url]
Clone (download) a repository that already exists on GitHub server,
including all the files, branches, and commits.
• Synchronize changes: Synchronize your local repository with the
remote repository on [Link].
$git fetch
Downloads all history from the remote tracking branches.
$ git push
Uploads all local branch commits to GitHub and sends changes to
the upstream remote repository from the local repository. Use this
 after a commit to push your changes to the upstream repository so
other developers can see your changes.
$ git pull
Updates your current local working branch with all new commits
from the corresponding remote branch on GitHub. git pull is a
combination of git fetch and git merge.
$ git rm
Removes a file or directory, while also updating the version control
record for the next commit. It is similar in use to the rm command in
Unix and should be used instead of filesystem commands to keep
version control history intact.
$ git mv
Renames or moves a file/directory, while also updating the version
control record for the next commit. It is similar in use to the mv
command in Unix and should be used instead of filesystem
commands to keep version control history intact.
$ git restore
Allows you to restore files from the Git index if deleted or
erroneously modified.
• Make changes: The following are a few Git commands used for
making changes, inspecting file evolution, and managing version
history within a Git repository. It covers actions such as comparing
content between branches, viewing file history, displaying metadata
and changes for a specific commit, adding files to version control,
and committing changes to the local repository with descriptive
messages. These commands are fundamental for tracking and
managing the development progress of a project using Git version
control:
$ git diff [first-branch]...[second-branch]
Shows content differences between two branches:
$ git log --follow [file]
 Lists version history for a file, including renames:
$ git log
Lists version history for the current branch:
$ git show [commit]
Outputs metadata and content changes of the specified commit:
$ git add [file]
Adds file revisions to version control, which can be either a new file
or modifications to an existing file:
$ git commit -m "[descriptive message]"
Saves changes in the working copy to the local repository. Before
running commit, register all your file changes by calling add, mv,
and rm on files that have been added, modified, renamed, or moved.
You also need to specify a commit message that can be done on the
command-line with the -m option.
• Rewrite history and redo commits: Rewriting branches, updating
commits and clearing history.
$ git rebase [branch]
Apply any commits of the current branch ahead of the specified one.
This command replays the commits from your current branch on the
upstream branch. This is different from merge in which the result is
a linear history rather than a merge commit, which can make the
revision history easier to follow. The disadvantage is that rebase
creates entirely new commits when it moves the history, so if the
current branch contains changes that have previously been pushed,
you are rewriting the history that the other clients may depend upon.
$ git reset [commit]
Undoes all commits after [commit], preserving changes locally:
$ git reset --hard [commit]
Clear staging area, rewrite working tree from specified commit.
Now that you have a basic understanding of some of the basic and advanced
Git commands, let us put this knowledge into practice.

Git command-line tutorial

Now, let us demonstrate how to use these commands, we will go through a
simple example to create a new local repository from scratch. For this
exercise, you should have a bash-like command shell on your system. This
is the default on most Linux systems as well as macOS. If you are on
Windows, you can do this via git bash.
If this is your first-time using Git, it is a good idea to put in your name and
email, which will be associated with all of your version control operations.
You can do this with the following commands:
git config --global [Link] "Put Your Name Here"
git config --global [Link] "your@[Link]"
After configuring your personal information, go to a suitable directory to
create your working project. First, create the project folder and initialize the
repository:
$ mkdir First_Demo
$ cd First_Demo
$ git init -b main ~/First_Demo
This creates the repository and initializes it so you can start tracking
revisions of files. Let us create a new file that we can add to revision
control:
echo "This is a sample file" > First_sample.txt
To add this file to revision control, use the git add command as follows:
$ git add First_sample.txt
You can add this file to version control by using the git commit command:
$ git commit First_sample.txt -m "First commit"
Congratulations on making your first command-line commit using Git! You
can double-check to make sure that your file is being tracked in revision
control by using the git log command, which should return output similar to
the following:
Author:neerajdevopsdemo <125783628+neerajdevopsdemo@[Link]
[Link]>
Date: Sun Feb 19 [Link] 2023 +0530
First commit
From the above log, you can get some details including branch information
(the default branch is master), and revisions by globally unique identifiers
(GUIDs).
You can do a lot more from the command-line but it is often easier to use a
Git client for your workflow or IDE integration designed for a developer
workflow. In the next sections, we will talk about these client options.

Git clients
Several open-source clients that you can use to work with Git are optimized
for various workflows. Most clients do not try to do everything, but
specialize in visualizations and functionality for specific workflows.
The default Git installation comes with a couple of handy visual tools that
make committing and viewing history easier. These tools are written in
Tcl/Tk, are cross-platform, and are easily launched from the command-line
to supplement the Git command-line interface (CLI). The following are
such tools:
• Gitk: gitk provides a way for navigating, viewing, and searching
the Git history of your local repository. The gitk user interface
displaying the history for the project is shown in Figure 3.10:
 Figure 3.10: Gitk application bundled with Git

The top pane of gitk displays the revision history with branching
information, which can be useful for check complicated branch
history. Below this are search filters that can be used to find
commits containing specific text. Finally, for the selected changeset,
you can see the changed files and a textual diff of the changes,
which is also searchable.
• Git-gui: The other tool that comes bundled with Git is git-gui.
Unlike git, which only shows information about the repository
history, git-gui allows you to modify the repository by executing
many of the Git commands, including commit, push, branch, merge,
and others. Figure 3.11 shows the git-gui user interface. On the left
side, changes to the working copy are shown, untracked and
unstaged changes are on top, and the files that will be included in
the next commit are on the bottom. The detailed content of the
selected file is shown on the right side. At the bottom right, buttons
are provided for common operations like rescan, sign off, commit,
and push. Further commands are available in the menu for advanced
 operations like branching, merging, and remote repository
management.

Figure 3.11: Git Gui application bundled with Git

• GitHub desktop: GitHub desktop is workflow-driven user

interface. This is the most popular third-party GitHub user interface,
and as mentioned earlier, also conveniently comes bundled with the
command-line tools. GitHub desktop is similar to git-gui, but is
optimized for integration with GitHub’s service, and the user
interface is designed to make it easy to follow workflows similar to
GitHub flow. The GitHub desktop user interface is shown in Figure
3.12:
 Figure 3.12: Git hub desktop client

In addition to the same sort of capabilities as git-gui to view changes,

commit revisions, and pull/push code, GitHub desktop has a bunch of
advanced features like commit attribution, image diff support, and syntax
highlighted differences that make managing your code much easier. GitHub
desktop can be used with any Git repo but has features tailored specifically
for use with GitHub-hosted repositories.
There are other popular Git tools in the market as well. A few of them are:
• Sourcetree: A Git client made by Atlassian. It is a good alternative
to GitHub Desktop and has only a slight bias toward Atlassian’s Git
service, Bitbucket.
• GitKraken client: A commercial Git client. It is free for open-
source developers but paid for commercial use.
A complete list of Git GUI clients is maintained on the Git website.
Git desktop clients are a great addition to the collection of available source
control management tools. However, the most useful Git interface may
already be inside your IDE.

Git IDE integration

Many integrated development environments (IDEs) include Git support
either as a standard feature or in the form of a well-supported plug-in. With
these integrations, you can make your favorite IDE to do basic version
control operations like adding, moving, and removing files, committing
code, and pushing your changes to an upstream repository.
One of the most popular Java IDEs is JetBrains IntelliJ IDEA. It has an
open-source community edition and a commercial version with additional
features for enterprise developers. The IntelliJ Git support is full-featured,
with the ability to sync changes from a remote repository, track and commit
changes performed in the IDE, and integrate upstream changes. Refer to the
following Figure 3.13:

Figure 3.13: IntelliJ tool

IntelliJ offers a rich set of features that you can use to customize the Git to
your team workflow.
Other Java IDEs where you can expect great Git support from, include the
following:
• Eclipse: Eclipse is a fully open-source IDE that has strong
community support and is run by the Eclipse Foundation. The
Eclipse Git support is provided by the EGit project, which is based
on JGit, a pure Java implementation of the Git version control
system.
• NetBeans: Offers a Git plug-in that fully supports workflow from
the IDE.
• Visual Studio Code: Supports Git along with other version control
systems out of the box.
• Oracle JDeveloper: While it doesn’t support complicated
workflows, JDeveloper does have basic support for cloning,
committing, and pushing to Git repos.

Git workflows
A Git workflow is a method or recommendation for how to use Git to
accomplish work in a consistent manner. Git workflows encourage
developers and DevOps teams to leverage Git effectively and consistently.
Git offers a lot of flexibility in how users manage changes. Git's focus is on
flexibility, so there is no standardized process on how to use Git. When
working with a team on a Git-managed project, it’s important to make sure
that the team collectively decides how the flow of changes will be applied.
To ensure the team is on the same page, an agreed-upon Git workflow
should be developed or selected. There are several publicized Git
workflows that may be a good fit for your team. Here, we will discuss some
of these Git workflow options.

Centralized workflow
The centralized workflow is a great Git workflow for teams transitioning
from Subversion (SVN). Like Subversion, the centralized workflow uses a
central repository to serve as the single point-of-entry for all changes to the
project. The default development branch is called main and all changes are
committed into this branch. This workflow does not require any other
branches besides main.
Transitioning to a distributed version control system may seem like a
daunting task, but you do not have to change your existing workflow to take
advantage of Git. Your team can develop projects in the exact same way as
they do with Subversion.
Figure 3.14 shows the centralized workflow with only branch that is Master
branch

Figure 3.14: Centralized workflow

However, using Git in your development workflow presents a few

advantages over SVN.
• First, it gives every developer their own local copy of the entire
project and each developer work independently. They can add
commits to their local repository and completely forget about
upstream developments until it's convenient for them.
• Second, it gives you access to Git’s robust branching and merging
model. Unlike SVN, Git branches are designed to be a fail-safe
mechanism for integrating code and sharing changes between
repositories. In centralized workflow developers push to and pull
from remote server-side hosted repository. Compared to other
workflows, the centralized workflow has no defined pull request or
forking patterns. A centralized workflow is generally better suited
for teams migrating from SVN to Git and smaller size teams.

Feature branching
The Git feature branch workflow becomes a must have when you have
multiple developers working on a particular feature without disturbing the
main codebase. The core idea is that all feature development should take
place in a dedicated branch instead of the main branch.
Imagine two developers working on two different features. Now, if both the
developers work from the same branch and add commits to them, it would
make the codebase a huge mess with lot of conflicts. To avoid this, the two
developers can create two separate branches from the master branch and
work on their features individually. When they’re done with their feature,
they can then merge their respective branch to the master branch without
having to wait for the other feature to be completed. Figure 3.15 is an
example of a Feature branching workflow with Master and Feature
branches. The Feature branch should be short lived to minimize the risk of
merge conflict.

Figure 3.15: Feature branching workflow

Git-flow workflow
The Git-flow workflow was first published in a highly regarded 2010 blog
post6 from and since then it has been widely used by organizations that
have a scheduled release cycle. The Git-flow workflow defines a strict
branching model designed around the project release. This workflow does
not add any new concepts or commands beyond what is required for the
feature branch workflow. Instead, it assigns very specific roles to different
branches and defines how and when they should interact.
In Git-flow, there are two long-lived branches, develop for development
integration, and master for final releases. Developers create branches from
the develop branch and work on new features, and integrate that with the
develop branch once complete. When the develop branch has all the
features necessary for a release, a new release branch is created out of
develop branch. No code related to new features is added into the release
branch. Only code that relates to the release is added to the release branch,
i.e., patches and bugfixes. Once the release branch has stabilized and is
ready for release, it is integrated into the master branch. Once on the master,
only hotfixes are applied, which are small changes managed on a dedicated
branch. These hotfixes also need to be applied back to the develop branch,
as well as any other concurrent releases that need the same fix. Figure 3.16
shows a sample diagram for git-flow:

Figure 3.16: Git-flow workflow

Forking workflow
The forking workflow is fundamentally different than the other workflows
discussed so far. Instead of using a single server-side repository to act as the
“central” codebase, it gives every developer a server-side repository. This
means that each contributor has not one but two Git repositories, a private
forked repository, and a master server-side main code repository. The fork
workflow is popular among teams who use open-source software.
The flow usually looks like this:
1. The developer forks the main repository. A copy of this repository is
created in their account.
2. The developer then clones the repository from their account to their
local system.
3. A remote path for the server-side repository is added to the
repository that is cloned to the local system.
4. The developer creates a new feature branch in their local system,
makes changes, and commits them.
5. These changes, along with the branch, are pushed to the developer’s
copy of the repository on their account.
6. A pull request from the branch is opened to the official repository.
7. The main repository’s manager checks the changes and approves the
changes to get merged into the main repository.

Trunk-based development
All the above-mentioned git flow approaches are alternatives to the feature
branch development model. When developers finish new work, they
must merge the new code into the main branch. Yet, they should not merge
changes until they have verified that they can build successfully. During
this phase, conflicts may arise if modifications have been made since the
new work began. In particular, these conflicts are increasingly complex as
development teams grow and the code base scales. The longer the feature
branch is in active development, the higher the likelihood of merge conflicts
with other features and maintenance going on in the master branch (or
trunk). Trunk-based development7 eases the friction of code integration.
Trunk-based development is a version control management8 practice where
developers merge small, frequent updates to a core “trunk” or main branch.
It is a common practice among DevOps teams and part of the DevOps
lifecycle since it streamlines merging and integration phases. In fact, trunk-
based development is a required practice of CI/CD. Developers can create
short-lived branches with a few small commits compared to other long-
lived feature branching strategies. As codebase complexity and team size
grow, trunk-based development helps keep production releases flowing.

Best practices for source control management

To get the most out of source control management, it is important to follow
a few best practices:

• Review changes before committing: Always make sure to review

the code before making commits, especially when you are
committing to a shared repository. Typically, code reviews are a part
of the delivery pipeline before merging code with the master branch
or a specific repository via a review system or a pull request. Code
reviews put a second eye on code modifications and help detect any
issues in the code. It also acts as a quality control mechanism that
helps organically improve the overall code quality while increasing
awareness of each code change.
• Commit often: Every commit is a snapshot of the codebase at a
particular point in time. Frequent commits allow you to
incrementally save completed changes to the codebase, which gives
you more opportunities to revert to an earlier state if a change
results in an error.
• Include a detailed commit message: A commit message should be
meaningful so other developers can get the context of a change
done. This can be helpful for troubleshooting if any issue arises
from particular changes. However, it does not mean that users
should create lengthy commit messages; instead, they should
describe the commit in a concise and meaningful manner. A good
 commit message can include details such as bug fix ID or
requirement ID, which reference the specifications of that code
change in project management platforms like Jira.
• Be sure you are working on the current version: With multiple
developers doing frequent code changes, your local copy of the
codebase can easily fall behind the master copy. Check out the latest
codebase before making changes to be sure you’re working with the
most up-to-date version.
• Standardize team workflows: Every developer can have their own
preferred processes on projects. But for SCM tools and procedures
to be effective, development teams need to establish consistent
workflows which creates the foundation for a better product.
• Use branches: Branches create separate lines of development to
allow developers to work in parallel on the same codebase without
impacting each other’s work. Branch strategy should be defined first
and to be used be used frequently. It’s good to create a new branch
for every change.

Conclusion
Good source control systems and practices lay the foundation for a solid
DevOps approach to building, releasing, and deploying code quickly. In this
chapter, we discussed what source control management is, what key
problems it solves, the history of source control systems, and explained why
the industry has moved to distributed version control. Then, we discussed
Code versioning and its best practices. We analysed which Source control
tool we should choose and why Git is suitable for your team.
We talked about Git components, and we did our first Git pull request. We
discussed a whole set of new command-line, desktop, and integrated tools
to work with Git repos. We elaborately discussed various Git client tools.
We had insights of different Git workflows. Toward the end of the chapter,
we discussed the best practices for source control management.
In the next chapter, we will discuss container and why we use it.
1. Source: [Link]
als#rcs---revision-control-system---first-generation
2. Source: [Link]
als#sccs---source-code-control-system---first-generation
3. Source: Google Trends
4. Source: [Link]
5. Git documentation: [Link]
6. Source: [Link]
7. Paul Hammant, a strong advocate for trunk-based development, has
set up a full website and written a book on the topic. Source : https://
[Link]/
8. Source: [Link]
rol

Join our book’s Discord space

Join the book’s Discord Workspace for Latest updates, Offers, Tech
happenings around the world, New Release and Sessions with the Authors:
[Link]
 CHAPTER 4
Containerization

Introduction
Currently, the use of containers in production and other environments is
expanding at an exponential rate. You will hear about their many
advantages, including their ability to start more quickly, enable better
DevOps practices, have a smaller footprint than virtual machines, so forth.
However, the creation of containers was necessary to address a fundamental
issue in the IT sector: applications that function in one environment but fail
in another due to a manual configuration error or a failure to install a
necessary package or piece of software.
In a nutshell, a container is a packaging of the application and all other
necessary components for application to function. A container has its own
isolated view of the file system and registry while it is running, and this
view is separate from both the host and other containers as well. Therefore,
a container-based application will always work the same way. To better
understand what a container is, let us first consider why we might use one.

Structure
In this chapter, we will discuss the following topics:
 • Understanding containers and their usage
• History of containers
• Difference between containers and virtual machines
• Key container and image terminologies
• Docker architecture
• Docker container orchestration using Kubernetes
• Docker on your machine
• Run Kubernetes locally
• Best image building practices

Objectives
By the end of this chapter, you will have a comprehensive understanding of
container technology, starting with the fundamental question of why
containers are used in modern software development. It traces the historical
evolution of containers, explores their distinctions from virtual machines,
and introduces key container and image terminologies. The chapter delves
into Docker architecture, explaining its components, and progresses to the
practical aspects of Docker container orchestration using Kubernetes.
Practical implementation is emphasized through guidance on setting up
Docker on individual machines and running Kubernetes locally. The chapter
concludes with insights into best practices for building optimized container
images, ensuring that you are well-equipped to navigate the dynamic
landscape of containerization with both theoretical knowledge and practical
proficiency.

Understanding containers and their usage

The issue of how to get software to run consistently when moved from one
computing environment to another is resolved by containers. This might
involve moving a virtual machine from a physical machine in a data centre
to a private or public cloud, moving a virtual machine from a staging
environment to a production environment, or moving a virtual machine
from a developer’s laptop to a test environment.
According to the docker's creator Solomon Hykes, issues arise when the
supporting software environment is not the same. Some weird thing will
happen when you test it using Python 2.7 and then run it in production using
Python 3. Alternately, you might rely on the behavior of one SSL library
version while installing a different one. You'll run your tests on Debian
while using Red Hat for production, and strange things will start to happen.
He continued, And it's not just different software that can cause issues. The
software must run on it, even if the storage and security policies are
different or the network topology is.
Simply put, a container is the entirety of a runtime environment, including
the application’s dependencies, libraries, other binaries, and configuration
files needed to run it. To remove differences in OS distributions and
underlying infrastructure, the application platform and its dependencies are
containerized.
A developer's local laptop, an on-premises data center, or even the cloud can
all be used to build, test, deploy, and redeploy applications using containers.
Let us now understand the key advantages that make containers a
cornerstone for agile development and efficient application deployment:
• Reduced overhead: As containers do not contain operating system
images, they consume fewer system resources than conventional or
hardware virtual machine environments.
• Expanded portability: Applications running in containers can be
easily deployed to various hardware platforms and operating
systems.
• More reliable performance: No matter where they are deployed,
DevOps teams know that applications running in containers will
function the same way.
• Higher effectiveness: Applications can be scaled, patched, or
deployed more quickly, thanks to containers.
 • Accelerated scaling: The same infrastructure can support many
more containers because containers lack the overhead of VMs, such
as separate OS instances. Since containers are portable, scenarios
involving quick scale-up and scale-down are made possible by their
quick start and stop capabilities.
• More effective application development: The use of containers
helps agile and DevOps initiatives speed up the development,
testing, and production cycles.

History of containers
Join us on a journey through time as we unravel the evolution of containers,
reshaping the landscape of computing. From the scarcity of computing
resources in the 1960s to the present-day significance of cloud-native
deployments, these narrative traces the milestones that led to the emergence
and widespread adoption of containers. This is a story of technological
leaps, collaborative initiatives, and the transformative impact of containers
on modern software development. We can classify the entire journey as
follows:
1. Scarce resources and early attempts (1960s-1970s): Computing
resources were generally extremely scarce and expensive (by today's
standards) in the 1960s and 1970s. Processes took a very long time
to finish (again, by today's standards), and it was typical for a
computer to be devoted for a very long time to a single task for a
single user. There have been attempts made to enhance the sharing
of computing resources and address the bottlenecks and inefficiency
caused by these restrictions. Sharing resources was necessary, but it
was not enough. Finding a way to share resources without
interfering with one another or having one person unintentionally
bring down the entire system for everyone became necessary.
Advanced virtualization technology has begun to trickle into both
hardware and software. We will start with the software advancement
known as chroot. To create a test environment for a different
distribution, the chroot system was created in 1979 during the
development of Unix V7. It works by moving a process's root
 directory and any children it has to a different location in the
filesystem.
2. Evolution of Container concepts (1980s-2004): Chroot was
included in the Berkeley Software Distribution (BSD) in 1982.
After two decades, in 2000, FreeBSD developed the idea and
released the more complex jail command and utility in FreeBSD 4.0.
The use of FreeBSD Jails enables administrators to divide a
FreeBSD computer system into a number of independent, smaller
systems, or jails, with the option of allocating an IP address to each
system and configuration. Just to note, Linux Vserver was introduced
in the year 2021. Linux VServer is a jail mechanism similar to
FreeBSD Jails that can divide resources (file systems, network
addresses, and memory) on a computer system. The Linux kernel
has been patched to implement this operating system virtualization.
Solaris containers, which combine boundary separation provided
by zones with system resource controls, were introduced in 2004.
These containers were able to use ZFS's snapshot and cloning
capabilities. The zones advanced us even further by granting an
application access to the entire system's hardware as well as full
user, process, and filesystem space.
3. Process containers and Linux containers (2006-2008): In order to
limit, account for, and isolate a group of processes’ resource usage
(memory, CPU, disk I/O, and network), Google introduced process
containers in 2006. Later, it was renamed to cgroups, which focused
on isolating and limiting a process's resource usage. Cgroups and
Linux namespaces were merged into the Linux kernel in 2008,
which prompted IBM to create Linux Containers (LXC). It was the
first most complete implementation of the Linux container manager.
4. Dawn of Docker and container standards (2013-2015): Now
things get even more interesting. Docker emerged in 2013. That
same year, Google offered its Let Me Contain That for You
(lmctfy) open-source project, which gave applications the ability to
create and manage their own subcontainers. From there, we saw the
use of containers explode—Docker containers specifically. Initially,
 Docker used LXC as its default execution environment, later
replacing container manager with its own library, libcontainer. The
Open Container Initiative (OCI) was introduced in June 2015. Its
objective is to develop open standards for container runtimes and
image specification. This group is a part of the Linux Foundation.
Although Docker is a major contributor, the company also listed
other participants in its announcement of the new organization,
including Apcera, Amazon Web Services (AWS), Cisco, CoreOS,
EMC, Fujitsu, Google, Goldman Sachs, HP, Microsoft, Huawei
Technologies, IBM, Intel, Pivotal Software, the Linux Foundation,
Mesosphere, Joyent, Rancher Labs, Red Hat, and VMware. There is
no doubt that the evolution of containers and the ecosystem around
them has advanced to the point where finding some common ground
will be advantageous to all parties involved. This is evidenced by the
significant amount of attention they have received.
5. Kubernetes and container orchestration (2015-2018): When the
OCI was established, Docker also declared its intention to donate its
runC runtime and base container format. In quick succession, the
Docker v2 Schema 2 image format, donated in April 2016, became
the basis for the OCI Image Format Specification, and runC became
the reference implementation for the OCI Runtime Specification.
Both of these specifications were made available in July 2017 as
version 1.0. Orchestration of these systems was also developing
quickly in tandem with developments in the container ecosystem.
One month after the OCI was established, Google unveiled
Kubernetes v1.0 in July 2015. In conjunction with this release, the
Cloud Native Computing Foundation (CNCF), a collaboration
between Google and the Linux Foundation, was established. A
crucial development by Google that was included in Kubernetes v1.5
in December 2016 was the creation of the Container Runtime
Interface (CRI), which provided the necessary level of abstraction
for the Kubernetes machine daemon, kubelet, to support additional
low-level container runtimes. The CRI-compatible runtime container
that Docker had created to integrate runC into Docker 1.11 was
contributed in March 2017 by Docker, a CNCF member as well.
 6. Contributions and Standardization (2018-2021): In 2018, with
over 27000 stars and one of the largest open-source communities,
over 1500 people contributed to the GitHub project for Kubernetes.
AWS with Elastic Kubernetes Service (EKS), Azure with Azure
Kubernetes Service (AKS), Google with Google Kubernetes
Engine (GKE), and Oracle with Container Engine for Kubernetes
were forced to offer managed Kubernetes services as a result of the
widespread adoption of Kubernetes. In addition, well-known
software providers like VMware, RedHat, and Rancher began
promoting management platforms built on Kubernetes. The CNCF
received yet another reference implementation from Docker in
February 2021. The distribution of images (pushing and pulling
container images) was the main focus of this contribution. Version
1.0 of the OCI Distribution Spec, based on the Docker Registry
HTTP API V2 protocol, was released by the OCI three months later,
in May 2021.
7. Present-day significance and cloud native deployments: In the
contemporary landscape, containers and orchestration tools play a
pivotal role in cloud native deployments. Today, cloud native
deployments frequently make use of containers and orchestration
tools like Kubernetes. Containers are crucial for maintaining
deployment flexibility across various hosts and scaling distributed
applications. Using shared infrastructure and pay-per-use storage,
cloud service providers like AWS, Google Cloud, Microsoft Azure,
and others are continuously expanding their product offerings.
Congratulations on successfully navigating through the History section! We
covered years of improvement and development in a few paragraphs. You
were introduced to a number of projects that later evolved into our
solutions, as well as some of the jargon used to describe containers and their
deployment. You have learned how much Docker has contributed to the
state of containers today. This is the ideal time to gain a thorough
understanding of the container ecosystem, the technical details underlying
containers, and the implementation components that are involved.
Before diving into that, let us first discuss the distinctions between virtual
machines and containers and how to choose one.

Difference between containers and virtual

machines
Both virtual machines and containers are used for creating isolated virtual
environments for developing and testing software or applications. How they
differ is the key question.

Understanding virtual machines

A virtual machine is a simulation or virtual representation, of a real physical
computer. Even when they are all running on the same host, each virtual
machine has its own operating system and operates independently of the
others. It is also known as a guest machine because it runs on the host's
hardware as a guest.
A VM cannot communicate with a physical host computer directly. To
communicate with the underlying physical hardware, it needs a thin layer of
software called a hypervisor.

Working of virtual machine

Virtualized hardware, an OS, and any necessary binaries and libraries are all
contained within the virtual machine, allowing it to run any application.
Virtual machines are, therefore, independent and have their own
infrastructure.
Refer to the following figure:
 Figure 4.1: Virtual machine

Each virtual machine is totally isolated from the host operating system.
Additionally, it needs its own OS, which may not be the same as the OS of
the host. Each has its own set of applications, libraries, and binaries.

Components of virtual machine

These are the components of a virtual machine:
• Hypervisor: It allocates physical computing resources (i.e.
processors, storage and memory) to each Virtual Machine.
• Host machine: It is the hardware on which the Virtual Machine is
installed.
• Guest machine: another name for the virtual machine (VM).

Pros and cons of virtual machine

Below are some of the pros of VM:
• VMs reduce expenses: Customers do not need to purchase a new
server every time they want to run a different OS because multiple
VMs can run on a single physical computer, allowing them to get
more use out of each piece of hardware they already own.
• Scalable: One host machine makes it possible to effectively manage
all the virtual environments thanks to the hypervisor's centralized
 power. You can install multiple system environments because these
systems are totally independent of one another. Multiple copies of
the same virtual machine can be quickly deployed using cloud
computing to better handle load increases.
• Portable: VMs can be moved as required among the physical
computers in a network. As a result, workloads can be distributed
among servers with extra processing power.
• Flexible: Because you can copy an existing VM with the OS already
installed, creating a virtual machine is quicker and simpler than
installing an operating system on a physical server. To handle new
tasks as they come up, developers and software testers can instantly
create new environments.
Now discuss some of the cons of VM:
• Unstable performance: If infrastructure requirement is not
adequate: If infrastructure requirements are not met, running
multiple virtual machines on one physical machine may lead to
unstable performance.
• Take lots of system resources: Virtual machines, which can be
many GBs in size, can consume a lot of the host computer's system
resources. Running a single application on a virtual server means
running a copy of the operating system as well as virtual copy of all
the hardware. As a result, many RAM and CPU cycles are quickly
accumulated.
• Relocation of an app: Due to its constant linkage to the operating
system, moving an application that is currently running on a virtual
machine can also be challenging. Therefore, you need to migrate
both the OS and the application. Additionally, when a virtual
machine is created, the hypervisor allots hardware resources
specifically for the VM.
Some popular VM providers are:
• VirtualBox
• Hyper-V
 • VMware vSphere

Understanding container
Containers are lightweight software packages that contain all the
dependencies required to execute the contained software application.
Virtualizing the software separates the software application from the host.
This enables users to create multiple workloads on a single OS instance.
The host operating system's kernel meets the needs of running an
application's different functions, which are separated into containers.
Isolated tasks are performed in each container. It cannot come in conflict
with other applications that are running in separate containers or harm the
host computer.

Working of containers
You can make a template of the necessary environment when working
inside a container. The container essentially runs a snapshot of the system at
a specific time, ensuring consistency in the behavior of an application.
All the individual applications running inside the container share the host's
kernel. Bins, libraries, and other runtime components are the only elements
that each container needs.
Refer to the following figure:
 Figure 4.2: Containers

Pros and cons of container

There are some of the pros of using containers:
• Speed: Since containers are lightweight, you can easily limit their
CPU usage and memory. They only include high-level software, so
they are very fast to modify and iterate.
• Excellent for CI/CD: Containers are also excellent for
implementing Continuous Integration and Continuous
Deployment (CI/CD). By sharing and merging images among
developers, they promote collaborative development.
• Resource efficiency and scalability: Containers share the host OS
kernel, allowing multiple containers to run on a single machine with
minimal overhead compared to virtual machines, improving resource
utilization. Also, Containers are lightweight and can be easily scaled
up or down, making them ideal for handling fluctuating workloads
and Microservices architectures.
Now, let us discuss some cons of using containers described as follows:
• The host’s kernel limits the use of other operating systems: A
container uses the host OS's kernel and is dependent on the host OS.
As a result, containers can be different from the underlying OS only
in terms of dependency and not type. The host's kernel puts limits on
using other operating systems.
• Shared host exploits: Containers still cannot provide the same level
of security and stability as VMs. They are less isolated than virtual
machines because they share the host's kernel. As a result, containers
are isolated at the process level, and one container may have an
impact on other containers by compromising the kernel's stability.
• Other issues: Additionally, after a container completes its task, it
shuts down and deletes all the data it contained. The data must be
saved using Data Volumes if you want it to stay on the host server.
This requires manual configuration and provisioning on the host.
Some popular container providers are:
• Docker
• LXD
• Hyper-V containers
• Java containers
• AWS
• Windows Server containers
• PodMan (RedHat)

Choosing VMs vs containers

Depending on the tasks you want your virtual environment to perform, you
should choose between virtual machines and containers.
Virtual machines are a better solution if you need to:
• Control multiple operating systems and applications from a single
server.
• Run an application program that requires all the resources and
features of an OS.
• Ensure complete isolation and security.
Containers are suitable if you need to:
• Make the most of a server's capacity by running as many
applications as possible.
• Deploy numerous copies of the same application.
• Have a lightweight, quick-starting system.
• Create an application that can be run on any infrastructure.

Key container and image terminologies

The world of containers has its own lexicon, and you will encounter the
following terms frequently:
• Container: The encapsulation of an application and all its required
dependencies and system resources running within an isolated
“space” on a host machine. Containers share the host machine’s
operating system and kernel, but utilize low-level features that allow
isolation between processes running inside the container and other
processes on the same host. Containers enable the portability of an
application or service between computing environments without the
risk of changes in behavior because of differing dependency sets.
• Container image: A container image is a static file with executable
code that can create a container on a computing system. A container
image is immutable—meaning it cannot be changed, and can be
deployed consistently in any environment. It encompasses all the
environment configuration and explicitly defines all the resources to
which a container will have access after it is launched. The image
shares the operating system kernel of the host, so it does not need to
include a full operating system.
• Base image: A base image is the image that is used to create all your
container images. Commonly used base images can describe a base
operating system and/or include a specific package or set of
dependencies. Your base image can be an official Docker image,
such as Centos, or you can modify an official Docker image to suit
your needs, or you can create your own base image from scratch.
An image based on another will specify the image it inherits from,
also known as a parent image, in the first line of the Dockerfile. A
parent image is not required to be a base image.
• Image ID: When an image is built, it is assigned a unique ID in the
form of a SHA-256 hash calculated from the contents of the image
metadata configuration file.
• Image digest: A unique ID in the form of a SHA-256 hash
calculated from the contents of an image manifest file.
• Image manifest: A JSON file that contains metadata about a
container image. It contains the image digests of the image metadata
configuration file and all the image layers.
 • Image layer: An image is made up of layers. Every layer is a "diff"
that records the changes that have been made to the image since the
previous layer was added. When building an image, the platform
creates a new layer for each instruction in the Dockerfile. The
container begins execution with the first instruction in the file and
proceeds to execute each instruction in order. The initial instruction
might install packages, while subsequent instructions might copy
files or create directories.
Beginning with a base layer, subsequent layers are stacked
sequentially, and each layer consists of a delta of changes from the
previous layer.
• Image tag: Using simple labels and aliases, you can describe an
image using image tags. Tags can be anything that can describe the
image, such as the project version, an image's features, or even just
your name.
A tag is unique to an image binary; however, an image binary can
have multiple tags. It assists you in managing the project's version
and enables you to monitor the entire development cycle.
• Container registry: A container registry is a repository—or
collection of repositories—used to store and access container
images. Often, you may hear the terms Docker registry and
container registry used interchangeably; however, be aware that a
container registry might not support all image formats specific to
both Docker and OCI images.
• Image repository (image name): An image repository is a collection
of related container images with different tags (versions) stored in a
container registry. It stores all the versions of an image, making them
available for distribution. The name of an image repository is
usually referred to as the image name.

Docker architecture
Docker is a true trademark of container. It has built an entire technology
stack around containerization. When you install Docker Desktop on your
development machine, you get more than just the ability to run containers.
You will receive an entire container platform that will make building,
running, and managing containers simple and convenient for developers.
It is critical to understand that installing Docker is not required for creating
container images or running containers. It is just a popular and convenient
tool for doing so. In the same way, you can package a Java project without
using Maven or Gradle, and build a container image without using Docker
or a Dockerfile. My advice to a developer new to containers is to start with
the toolset Docker provides and then experiment with other options or
methods to get a good feel for a comparison. Even if you choose to use
other tools instead of or in addition to Docker, a lot of time and effort was
put into engineering a good developer experience, and this alone is worth it.
With Docker, you can create an isolated environment where a user or
application can run while sharing the OS and kernel of the host system
without interfering with the functionality of another isolated environment
running on the same system (a container).
Docker lets you do the following:
• Define a container (an image format)
• Create a container image
• Manage container images
• Distribute/distribute container images
• Establish a container environment;
• Start/run a container (a container runtime); and
• Manage the lifecycle of container instances.
The container landscape contains much more than Docker, but many of the
container toolset alternatives focus on a subset of these items. Beginning
with learning how Docker operates is helpful in understanding and
evaluating these alternatives.
Many pictures and diagrams are readily available that describe the Docker
architecture. Figure 4.3 illustrates how Docker works on your development
machine. Docker client calls Docker host to run a container from a
particular image. If the Docker host has that image, it runs a container from
it. If it does not have the image, it looks for it in the registry, pulls it on the
host, and then runs a container from it. Note that any of these three
components can be on the same machine or on different machines.

Figure 4.3: Docker architecture

Dockerfile
Docker can automatically create images by following the directions in a
Dockerfile. A Dockerfile is a text file that includes every command a user
could enter on the command line to put together an image. Hence, it
contains a set of instructions used to build a Docker image. It defines the
environment in which the application runs, specifies the base image, copies
files, installs dependencies, and executes commands needed to prepare the
final containerized application. When you build a Docker image, Docker
reads the Dockerfile to automate the image creation process.

Sample Dockerfile
The following is a sample of a Dockerfile to let you understand what it
looks like:
FROM tomcat:8-jre11
LABEL “Project”=”Docker-Demo”
LABEL “Author”=”Neeraj”
RUN rm -rf /usr/local/tomcat/webapp/*
COPY target/[Link] /usr/local/tomcat/webapp/[Link]
EXPOSE 8080
CMD [“[Link]”, “run”]
WORKDIR /usr/local/tomcat/
VOLUME /usr/local/tomcat/webapp

Docker container orchestration using Kubernetes

There are many tasks that need to be managed in a Docker cluster
environment, including scheduling, communication, and scalability. Any
orchestration tool on the market can handle these tasks, but some of the
most well-known ones are Docker Swarm, a native orchestration tool for
Docker, and Kubernetes, one of the fastest-growing open-source projects
ever.
Kubernetes is an open-source container management (orchestration) tool.
Deploying containers, scaling and descaling them, and balancing container
loads are all part of its container management duties. AWS ECS, Azure
AKS, EKS, and GCP GKE are just a few of the cloud-based Kubernetes
services known as Kubernetes as a Service (KaaS) that are accessible.
Refer to the following figure:
 Figure 4.4: High-Level view of Kubernetes Architecture

Working of Kubernetes
There is a master node and a number of worker nodes in Kubernetes, and
each worker node can manage a number of pods. A number of containers
arranged in a group to form a functional unit are known as pods. Pods can
be used to begin designing your applications. Once your pods are ready, you
can tell the master node which ones you want to deploy and how many.
From this point, Kubernetes is in control. The pods are taken and deployed
to the worker nodes. If a worker node goes down, Kubernetes starts new
pods on a functioning worker node. To improve the application and increase
customer satisfaction, Kubernetes makes it simple to build new features.
Master node components: In Kubernetes, a master node refers to a control
plane component responsible for managing and overseeing the cluster’s
operations such as scheduling, detecting and responding to cluster events. It
acts as the brain of the cluster, coordinating and maintaining the desired
state of the system. The main components of a Kubernetes Master node
includes:
• API server: The API server serves as the primary interface for
cluster management. It exposes the Kubernetes API, which allows
users and other components to interact with the cluster, create and
 manage resources, and perform operations such as scaling and
deployment.
• etcd: etcd is a distributed key-value store used for storing and
replicating the cluster’s configuration and state data. It provides
reliable and consistent storage, ensuring the availability and
consistency of the cluster’s data. It also provides a REST API for
CRUD operations.
• Controller-manager: The controller manager consists of various
controllers that handle different aspects of the cluster’s behaviour. It
includes controllers for managing replication, endpoints, services,
namespaces, and other higher-level abstractions. The controller
continuously monitor the cluster’s state and work to reconcile the
desired state with the current state.
• Scheduler: The scheduler is responsible for placing pods onto nodes
based on resource requirements, affinity rules, and other policies. It
examines the cluster’s resources and constraints to determine the
optimal placement of pods for efficient resource utilization.
Worker node components: In Kubernetes, a worker node, also known as
worker or a minion, is a component responsible for running the containers
and executing the task assigned to the cluster. It hosts the actual workload
and communicates with the master node to receive instructions and report
the status of the tasks. Node components run on every node, maintain
running pods and provide them the Kubernetes runtime environment. The
main components of Kubernetes worker node include:
• kubelet: The kubelet is an agent that runs on each worker node and
is responsible for managing the containers on that node. It
communicates with the master node to receive instruction about
which containers to run, their configuration, and their desired state.
The kubelet ensures that the containers are running, monitors their
health, and reports the status back to master.
• Kube-proxy: The kube-proxy is a network proxy that runs on each
worker node. It is responsible for routing network traffic to the
appropriate containers and services within the cluster. Kube-proxy
 helps enable service discovery and load balancing by implanting the
necessary network rules and forwarding rules.
• Container runtime: The container runtime is the software
responsible for running containers. Kubernetes supports multiple
container runtimes, such as Docker, containerd, and CRI-O. The
container runtime interacts with the host operating system to start,
stop, and manage containers based on the instructions provided by
the kubelet.

Docker on your machine

Containers are based on a combination of existing Linux features, while
their implementation differs in terms of detail. In a sense, however, a
container image is simply a tar disk of a complete filesystem, and a running
container is a Linux process that is restricted to a level of isolation from
other processes running on the host. The implementation of a Docker
container, for example, primarily involves these three ingredients:
• Namespaces
• cgroups
• A union filesystem
However, on your local filesystem what does the container look like? Let us
get a look at where the Docker is storing things on our development
computer first. Let us look at a real Docker image pulled from Docker Hub
to see what this is all about.
After installing Docker Desktop on your machine, run the below command
from a terminal:
docker info
Detailed information on installation will be provided by the above
command. The output includes information about where your images and
containers are stored with the label Docker Root Dir.
The following example output indicates that the Docker root directory is
/var/lib/docker:
C:\Users\Dell>docker info
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.10.4
Path: C:\Program Files\Docker\cli-plugins\[Link]
compose: Docker Compose (Docker Inc.)
Version: v2.17.3
Path: C:\Program Files\Docker\cli-plugins\[Link]
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.0
…..
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 23.0.5
Storage Driver: overlay2
 Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk
syslog
Swarm: inactive
Runtimes: [Link].v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 2806fc1057397dbaeefbea0e4e17bddfbd388f38
runc version: v1.1.5-0-gf19387a
init version: de40ad0
Security Options:
seccomp
Profile: builtin
Kernel Version: [Link]-microsoft-standard-WSL2
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 3.764GiB
Name: docker-desktop
ID: 8172611d-ca21-4e67-ad2a-06800809bf94
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: [Link]
HTTPS Proxy: [Link]
No Proxy: [Link]
Registry: [Link]
Experimental: false
Insecure Registries:
[Link]
[Link]/8
Live Restore Enabled: false
Now, let us pull a Docker image by using the command docker pull
IMAGE NAME and see what it looks like on the filesystem:
$ docker pull openjdk
Using default tag: latest
latest: Pulling from library/openjdk
5a581c13a8b9: Pull complete
26cd02acd9c2: Pull complete
66727af51578: Pull complete
Digest: sha256:05eee0694a2ecfc3e94d29d420bd8703fa9dcc64755962e267f
d5dfc22f23664
Status: Downloaded newer image for openjdk:latest
[Link]/library/openjdk:latest
You can check the newly pulled image on Docker Desktop as well. Refer to
the following image:

Figure 4.5: Docker Desktop

The one we pulled in the previous command brought in the image with the
tag latest. That’s the default setting, but we can specify an openjdk image
version like this:
C:\Users\Dell>docker pull openjdk:11-jre
11-jre: Pulling from library/openjdk
001c52e26ad5: Pull complete
d9d4b9b6e964: Pull complete
2068746827ec: Pull complete
8510da692cda: Pull complete
b6d84395b34d: Pull complete
bf03fea6c3ad: Pull complete
Digest: sha256:356949c3125c4fa8104745e7ea92bd995da4567634e6599b47
0d2f972d13e0e2
Status: Downloaded newer image for openjdk:11-jre
[Link]/library/openjdk:11-jre:
Now, the command docker images lists all of the images stored locally. You
can see from its output that two versions of the openjdk image are stored.
C:\Users\Dell>docker images

REPOSITOR TAG IMAGE ID CREATED SIZE

Y

[Link]/k8s-mi v0.0.39 67a4b1138d2 3 weeks ago 1.05GB

nikube/kicba d
se

openjdk latest 71260f256d1 2 months ago 470MB

9

openjdk 11-jre 362cda5d270 8 months ago 302MB

e

You can learn more details about the latest openjdk image by running the
docker inspect command using the image ID:
C:\Users\Dell>docker inspect 71260f256d19
This is what the output will look like:
[
 {
“Id”: “sha256:71260f256d19f4ae5c762601e5301418d2516ca591103b1
376f063be0b7ba056”,
“RepoTags”: [
“openjdk:latest”
],
“RepoDigests”: [
“openjdk@sha256:9b448de897d211c9e0ec635a485650aed6e28d4ec
a1efbc34940560a480b3f1f”
],
“Parent”: “”,
“Comment”: “”,
“Created”: “2023-02-08T[Link].323166355Z”,
“Container”: “7cb9d6257345f1d39107682b018491eb5d3f38dbe81b79
c7a9b4cd1ae90de638”,
“ContainerConfig”: {
“Hostname”: “7cb9d6257345”,
“Domainname”: “”,
“User”: “”,
“AttachStdin”: false,
“AttachStdout”: false,
“AttachStderr”: false,
“Tty”: false,
“OpenStdin”: false,
“StdinOnce”: false,
 “Env”: [
“PATH=/usr/java/openjdk-18/bin:/usr/local/sbin:/usr/local/bin:/us
r/sbin:/usr/bin:/sbin:/bin”,
“JAVA_HOME=/usr/java/openjdk-18”,
“LANG=[Link]-8”,
“JAVA_VERSION=[Link]”
],
“Cmd”: [
“/bin/sh”,
“-c”,
“#(nop) “,
“CMD [\”jshell\”]”
],
“Image”: “sha256:ac2e438abda50fce392b0843eb4e5436188c82e2ed
3384069d21795bf6c200c0”,
“Volumes”: null,
“WorkingDir”: “”,
“Entrypoint”: null,
“OnBuild”: null,
“Labels”: {}
},
“DockerVersion”: “20.10.12”,
“Author”: “”,
“Config”: {
 “Hostname”: “”,
“Domainname”: “”,
“User”: “”,
“AttachStdin”: false,
“AttachStdout”: false,
“AttachStderr”: false,
“Tty”: false,
“OpenStdin”: false,
“StdinOnce”: false,
“Env”: [
“PATH=/usr/java/openjdk-18/bin:/usr/local/sbin:/usr/local/bin:/us
r/sbin:/usr/bin:/sbin:/bin”,
“JAVA_HOME=/usr/java/openjdk-18”,
“LANG=[Link]-8”,
“JAVA_VERSION=[Link]”
],
“Cmd”: [
“jshell”
],
“Image”: “sha256:ac2e438abda50fce392b0843eb4e5436188c82e2ed
3384069d21795bf6c200c0”,
“Volumes”: null,
“WorkingDir”: “”,
“Entrypoint”: null,
 “OnBuild”: null,
“Labels”: null
},
“Architecture”: “amd64”,
“Os”: “linux”,
“Size”: 469930470,
“VirtualSize”: 469930470,
“GraphDriver”: {
“Data”: {
“LowerDir”: “/var/lib/docker/overlay2/62f68265846683df6ece73
be75d4a0b4da9836d1691cd0456411a8c0907995c1/diff:/var/lib/docker/over
lay2/4a6910b44a0c6dbd3a2c248536a3b5c7ebd47977bcf850b15bbd319a1e
544727/diff”,
“MergedDir”: “/var/lib/docker/overlay2/7cd0ecdb1d600f96d8814
bc9354665dfc13110aee62cb38e5065cd33fa52e2a8/merged”,
“UpperDir”: “/var/lib/docker/overlay2/7cd0ecdb1d600f96d8814b
c9354665dfc13110aee62cb38e5065cd33fa52e2a8/diff”,
“WorkDir”: “/var/lib/docker/overlay2/7cd0ecdb1d600f96d8814bc
9354665dfc13110aee62cb38e5065cd33fa52e2a8/work”
},
“Name”: “overlay2”
},
“RootFS”: {
“Type”: “layers”,
“Layers”: [
 “sha256:9cd9df9ffc972e9abc946d855162ef0c40dff9a89f10c962b
7920154a3d943d8”,
“sha256:077bff59ce5723e3c7d78bdf4fd8b10d72f6f8474b97cdb9
323816aa5d8314a6”,
“sha256:56285d9a776094205dc0b66078bf0719f50c734a0075429
2e6fcbd13b17f5155”
]
},
“Metadata”: {
“LastTagTime”: “0001-01-01T[Link]Z”
}
}
Numerous useful details are produced by the docker inspect command. The
GraphDriver section, which contains the paths to the folders where all the
layers that are a part of this image reside.
Layers in a Docker image are created according to instructions in the
Dockerfile that was used to create the image initially. In order to save space,
these layers can be translated into directories and shared across images.
Keep in mind the sections labelled LowerDir, MergedDir, and UpperDir.
All of the layers or directories that were used to build up the original image
can be found in the LowerDir section. These are read-only. The UpperDir
directory contains all of the content that has been modified while the
container is running. If any changes are required, a read-only layer in the
LowerDir is copied into the UpperDir where it can be written to. And this
is known as a copy-on-write operation.
Keep in mind that the data in UpperDir is ephemeral data that exists only as
long as the container lives. Actually, if you have data that you want to keep,
you should use Docker’s volume features to mount a location that will stick
around even after the container dies. A database-driven application, for
instance, will most likely use a volume mounted to the container to store its
database data.
At last, the MergedDir section combines everything from LowerDir and
UpperDir into a sort of virtual directory. Any edited layers that were copied
into UpperDir will overlay any layers in LowerDir due to the way the
Union File System operates.
The same image can be used to launch any number of containers. The image
blueprint will be used to create each container, which will run
independently.
Without having to be recreated, containers may be stopped and then
restarted. The docker ps -a command can be used to list all containers on
your system. You should be aware that the -a flag will show both running
and stopped containers:
C:\Users\Dell>docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORT NAME
S
96e678d1d77b [Link]/k8s-minikube/kicbase:v0.0.39 “/usr/local/bin/entr…”
3 hours ago Exited (255) 32 minutes ago [Link]:32772->22/tcp minikube
You can find a subdirectory named containers if you navigate to the Docker
root directory. Additional subdirectories within this directory will be named
after the container IDs of the various containers on your system. Containers
that have been stopped will keep their state and data in these directories so
they can be restarted if necessary. The associated directory will be deleted
when a container is deleted using the docker rm CONTAINER NAME
command.
Always remember to remove—not just stop—unused containers from your
systems on a regular basis. The old containers were stopped each time new
images were released, and new containers were launched based on the new
images. This oversight rapidly consumed hard drive space and ultimately
stopped new deployments. To remove unused containers in bulk, use the
following Docker command:
docker container prune

Basic tagging and image version management

After working with images a while, you will see that identifying them and
versioning them are a bit different from the way you version your file. In a
typical file system, with using file tags, you can give a file multiple
keywords so that, for example, every file that has the keyword “linux” in its
tag will show up when you search for it. We do not use Docker tags in that
manner. Instead of being used for categorization or keyword-based searches,
Docker tags are applied to images to provide meaningful labels, such as
version numbers or environment names which can help to track and manage
different builds or versions of the same image.
If not thoroughly understood, image versioning can prove to be challenging.
There are no guardrails (not the kind that Java developers are familiar to).
Instead of enforcing good practices, flexibility in image tagging is preferred.
It is preferable to get off to a good start with a naming and versioning
scheme that makes sense and adheres to an accepted pattern.
Let us say you failed to tag a file when you uploaded an Ubuntu image to
Docker Hub. This would be similar to the Download Ubuntu button on
Canonical’s website linking directly to the [Link]- You would not be
aware of the Ubuntu version you were downloading. Is it a version.02 or.10
release? Is this even the most recent version? Because of this, Ubuntu files
are frequently marked with names like [Link] so
that you can be sure of what you are getting. Similar to the 16.0.1-server-
amd64 section of the download file, a Docker tag should specify precisely
which image you are downloading from the repository.
Now, take the other example. The simplest command for pulling the
openjdk Docker image might take the following form:
docker pull openjdk
In actuality, what does this command give us? Do you not have several
versions of the OpenJDK images that can be used? Indeed, yes, and if you
care about having repeatable builds, you’ll recognize right away that this
ambiguity could be a problem.
The image tag, which represents a version, must be first included in this
command. We will pull version 11 of the openjdk according to the below
command:
docker pull openjdk:11
So, if not 11, what was I pulling before? If a tag is not specified, a special
tag called latest is implied by default. Although the tag is meant to direct
users to the most recent version of the image, this may not always be the
case. A tag may be changed at any time to point to a different version of an
image, and occasionally the tag latest may not even be set to point to
anything.
It is simple to make mistakes with the nomenclature as well, particularly
with tags, which have multiple meanings. The word “tag” can refer to a
particular version or the complete image tag, which combines all the
elements of identification, including the image name.
Here is the complete format of a Docker image tag with all possible
components:
[ registry [ :port ] / ] name [ :tag ]
The name of the image, also known as the image repository, is the only
element that must be present. If not specified, Tag is assumed to be latest.
Docker Hub is the default registry if the registry is not specified. To
reference an image on a registry other than Docker Hub, use the command
shown below:
docker pull [Link]/openjdk:11

Adding tags to images

Because the necessary tools have been integrated into the docker command
by the Docker developers, adding a tag to your image is quite
straightforward.
For instance, you customized your Ubuntu image to meet your needs
exactly. We will say that your Ubuntu image is version 1 and intended for
development. Let us develop a tagging system that uses that data. We could
use a tag similar to:
dev:v1.4.15
Anyone who is familiar with your versioning system will be able to tell that
this is version 1, created on April 15th. However, this could be problematic
for anyone unfamiliar with your versioning system; you could clarify this by
using language like:
dev:v1.5.15.2023
Although it illustrates the point, the above assumes a date notation, so even
with that, you could get into trouble.
How are your images tagged? Let us assume we are choosing the second of
the two options. But this also not appropriate. So, we will use the following
command to tag an image:
docker tag IMAGE ID image/TAG
TAG is our newly developed versioning tag, and IMAGE ID is the 12-
character identification string for the image (listed from the Docker images
command). So, the following would be our command to tag the Ubuntu
image:
docker tag 7b9b13f7b9c0 ubuntu/dev:v1.5.15.2023

Image layers
It is crucial to have a solid understanding of layers in order to build effective
containers. The details behind how you build the source of your containers
—your container images—greatly impact their size and performance, and
some approaches have security implications, making this concept even more
important to master.
Docker image layers are a fundamental concept in Docker, and they
represent the building blocks of Docker images. A Docker image layer is a
read-only file system layer that contains changes to the underlying file
system. Each layer is immutable and represent a specific set of changes to
the file system. Basically, Docker images are built by establishing a base
layer and then subsequently making small changes until you arrive at your
desired final state. Each layer represents a set of changes including, but not
limited to, the creation of users and related permissions, modifications to
configuration or application settings, and updates to existing packages or
adding/removing packages. These changes all amount to additions,
modifications, or the removal of sets of files in the resulting filesystem. For
example, let us say that you have a Docker image that contains an
application and its dependencies. The image might consist of multiple
layers, with each layer representing a different aspect of application. The
first layer might contain the base operating system and the second layer
might contain application code. Each subsequent layer builds on top of
previous layer, adding new changes to the file system.
Docker image layers are important for number of reasons:
• Image layers enable Docker to perform efficient storage and
transmission of images. When you pull a Docker image from
registry, Docker only needs to download the layer that have changed
since the last time the image was pulled. This makes the download
process faster and more efficient.
• Docker image layers enable Docker to perform efficient caching.
When you build a Docker image, Docker caches each layer
separately. This means that if you make a change to the Dockerfile
and rebuild the image, Docker can reuse the cached layer, only
rebuilding the layers that have changed. This makes the build
process faster and more efficient.
 • Finally, Docker image layers enable you to create smaller and
efficient images. By breaking down an image into multiple layers,
you can reuse common layers across multiple images, reducing the
overall size of each image making it easier to deploy and manage
your applications.

Run Kubernetes locally

There are numerous open-source platforms that you can test out to run
Kubernetes locally. We’ll go over one method for running Kubernetes
locally on a Windows computer in particular.
The most well-known and widely used option for running a Kubernetes
environment locally on a computer is Minikube. Minikube’s main objectives
are to support all Kubernetes features that are appropriate and to be the best
tool for developing local Kubernetes application.
To install the latest minikube stable release on x86-64 Windows using .exe
download:
1. Download and run the installer for the latest release. Or if using
PowerShell, use this command:
New-Item -Path ‘c:\’ -Name ‘minikube’ -ItemType Directory -Force
Invoke-WebRequest -OutFile ‘c:\minikube\[Link]’ -Uri ‘http
s://[Link]/atalinas/minikube/releases/latest/download/minikube-
[Link]’ -UseBasicParsing
2. Add the [Link] binary to your Path. Make sure to run
PowerShell as Administrator.
$oldPath = [Environment]::GetEnvironmentVariable(‘Path’, [Enviro
nmentVariableTarget]::Machine)
if ($[Link](‘;’) -inotcontains ‘C:\minikube’){ `
[Environment]::SetEnvironmentVariable(‘Path’, $(‘{0};C:\minikub
e’ -f $oldPath), [EnvironmentVariableTarget]::Machine) `
}
Note: If you used a terminal (like powershell) for the installation,
please close the terminal and reopen it before running minikube.

3. Start your cluster

From a terminal with administrator access (but not logged in as
root), run:
minikube start

Best image building practices

When it comes to building Docker images, there are some best practices that
you should keep in mind. Here are a few of them:
• Use .dockerignore file wherever its required:
You do not want to have certain things in your production Docker
image—things like your development environment configuration,
keys, your .git directory, or other sensitive hidden directories. When
you run the command to build a Docker image, you provide the
context, or the location of files you want to make available to the
build process.
Let us see the below example:
FROM ubuntu
WORKDIR /mydemo
COPY . /mydemo
EXPOSE 8080
ENTRYPOINT [“[Link]”]
See the COPY instruction? Depending on what you sent in as the
context, this could be problematic. It could be copying everything
from your working directory into the Docker image you build, which
will end up in any container launched from this image.
Make sure to use a .dockerignore file to exclude files from the
context that you do not want showing up unintentionally. You can
 use it to avoid accidentally adding any user-specific files or secrets
that you might have stored locally. In fact, you can greatly reduce
the size of the context (and the time it takes to build) by excluding
anything the build does not require access to:
# Ignore these files in my project
[Link]
.cache
*.log
.git
.cache
• Use version control system:
Just like any other code, Docker image should be versioned and
managed using a version control system (VCS) such as git. This
makes it easy to track changes, collaborate with others, and roll back
to previous versions if needed.
• Keep your images small:
One of the biggest advantages of using Docker is the ability to create
lightweight and portable images. To achieve this, it is important to
keep your images as small as possible by removing unnecessary
files, using slim base image, and avoid installing unnecessary
packages.
Utilize multistage builds to keep your images small. A multistage
build can be set up by creating a Dockerfile that uses multiple
FROM statements, which begin a build stage with a different base
image. By using multistage builds, you can avoid including things
like build tools or package managers that are not needed in a
production image.
For example, the following Dockerfile shows a two-stage build. The
first uses a base image that includes Maven. After the Maven build
 is complete, the required war file is copied to the second stage,
which uses an image that does not include Maven:

## 1st Build
FROM openjdk:8 AS BUILD_IMAGE
RUN apt update && apt install maven -y
RUN git clone [Link]
it
RUN cd docker_demo && mvn install

## 2nd Build
FROM tomcat:8-jre11
RUN rm -rf /usr/local/tomcat/webapps/*
COPY –from=BUILD_IMAGE app-repo/target/[Link] /usr/loca
l/tomcat/webapps/[Link]
EXPOSE 8080
CMD [“[Link]”, “run”]
This is a good way to implement using a custom image, which has
been reduced to only the bare minimum necessary to run your
application (including a shell).
• Use the COPY command carefully:
For each file or directory that you copy using the COPY command,
a new layer is added to your image. Use the COPY command
sparingly and avoid copying extraneous files if you want to reduce
the number of layers in your image.
• Keep your Dockerfile simple:
It is simpler to read, comprehend, and maintain a straightforward
Dockerfile. In your Dockerfile, stay away from complex commands
and multiple RUN statements, and use the fewest number of layers
possible.
Conclusion
Containers have revolutionized the way we develop, deploy, and manage
applications. They provide a lightweight and efficient solution for running
applications in different environments while ensuring consistency and
portability.
In this chapter, we began the exploration by dissecting the fundamental
concepts, distinguishing containers from virtual machines, and tracing their
evolutionary journey through the annals of history. The discourse expanded
to encompass critical container and image terminologies, laying a robust
groundwork for a nuanced comprehension of these transformative
technologies.
Continuing our journey, we navigated through the intricate architecture of
Docker and delved into the orchestration prowess of Kubernetes, unraveling
the complexities of this influential tool. The narrative then shifted towards
practical implementation, guiding you through the hands-on process of
setting up Docker on individual machines and executing fundamental
commands. Essential topics such as tagging, versioning, and the
significance of image layers were thoroughly explored. Subsequently, the
chapter provided valuable insights into running Kubernetes locally. Towards
the end of the chapter, we had insights of best Image build practices.
In the next chapter, we will discuss about continuous integration and how
CI/CD can drastically improve developers’ productivity.

Join our book’s Discord space

Join the book’s Discord Workspace for Latest updates, Offers, Tech
happenings around the world, New Release and Sessions with the Authors:
[Link]
 CHAPTER 5
Continuous Integration

Introduction
In Chapter 3, Manage Your Source Code, source control and a shared code
repository were discussed. After you have organized and chosen your
source control solution, you still need to take a few more steps to get to the
point where your users can enjoy the ideal user experience of your
delivered software.
As an individual developer, consider the steps you would take to advance
your software through every stage of the software development lifecycle.
The actual lines of code and associated unit tests would then be added to the
codebase after the acceptance criteria for a specific feature or bug fix for
your software have been established. Then, you would compile and run
each unit test to make sure your new code behaves as you anticipate (or, at
the very least, as defined by your unit tests) and does not impair known
existing functionality. When all tests are found to pass, you build and
package your application and perform integration tests in a quality
assurance (QA) environment to verify functionality. After receiving the
all-clear from your well-maintained test suites, you would finally deliver
and/or deploy your software to a production environment.
However, software development rarely goes as planned. When you start
working on a bigger project with a team of developers, strict
implementation of the ideal workflow is too simple. There are several new
issues that can block up the software delivery lifecycle's workings and
disrupt your schedule. This chapter explains how using Continuous
Integration and the associated best practices and toolkits can help you avoid
or lessen the most frequent roadblocks and headaches that software
development projects face along the way to completion.

Structure
In this chapter, we will discuss the following topics:
• Continuous integration
• Continuous integration in development team
• Critical concepts of continuous integration
• Key practices of continuous integration
• Declarative build script
• Continuous build
• Test automation
• Monitor and maintain tests
• Building pipeline with Jenkins

Objectives
This chapter aims to provide a comprehensive understanding of continuous
integration (CI) and its role within development teams. It will delve into
the critical concepts and key practices of CI. The chapter will emphasize the
importance of a declarative build script using popular tools like Ant, Maven,
and Gradle. By the end of this chapter, we will explore the nuances of
continuous build processes, shedding light on the significance of test
automation and the ongoing monitoring and maintenance of software
testing. Lastly, it will guide readers through the construction of a robust
build pipeline with Jenkins, offering practical insights into implementing
effective CI strategies for enhanced software development workflows.

Continuous integration
CI is a software development technique where developers frequently
integrate their work into a project's main source code repository. Submitting
your unique work (modified code) to the shared workspace (the potential
software solution) is known as integration. Technically, this is
accomplished by merging your individual work (personal branch) with the
common work area (Integration branch).
CI is required to expose integration-related issues as early as possible
through frequent and regular builds. The fundamental principal of CI is that
by frequently integrating changes, developers can find bugs earlier in the
process and spend less time trying to pin when and where a problem first
appeared. The longer a bug goes unnoticed, higher the chance of it
becoming deeply entrenched in the surrounding codebase.
From a development standpoint, it is much simpler to find, catch, and fix
bugs as soon as they are introduced rather than trying to extract them from
code layers that have already promoted to later stages of the delivery
pipeline. Bugs that are not found until the latest acceptance phases, and
especially those that make it all the way to release, result in more money
being spent on fixes and less time being spent on new features. In many
cases, it is now necessary to patch existing deployments in addition to
including and documenting the fix in a new version when fixing a bug in
production. This inherently cuts down on the amount of time the team must
work on creating new features.
It is critical to realize that implementing a CI solution does not guarantee
bug-free software. CI is merely an additional layer of security to remove the
most evident bugs from a release. It is true that a successful CI solution
involves much more than just coordinating code contributions to a common
repository and adhering to a mandate to integrate at predetermined
intervals. The components of a comprehensive, workable CI solution that
will simplify and speed up the software development process are described
in the sections that follow.

Continuous integration in development team

The software development process always impacts the business. The
promises a business has made to its customers are impacted by the quality
of the code, the design, the amount of time spent on development, and the
planning of features. Developers are supported by CI in assisting
businesses. You may have already understood the advantages of
implementing CI while reading the earlier topics. Let us examine some of
CI's advantages:
• No more lengthy integrations: You have a better chance of
spotting integration errors early if you build and integrate every
small change you make to your code. It is better to integrate
frequently and avoid the merge hell than to wait six months (as in
waterfall model) for integration and then spend weeks trying to fix
merge issues. When you check in your code, a CI tool like Jenkins
automatically builds and integrates it.
• Ready-for-production features: CD enables you to release
deployable features whenever you want. This is greatly
advantageous from a business standpoint. Within two to four weeks,
the features are designed, implemented, and tested, and they are
ready to go live with just a click of a button.
• Reporting and analysis: How often do the releases occur? What
percentage of builds are successful? What is the primary factor
leading to build failures? Making crucial decisions always requires
access to real-time data. Recent data is always needed for projects in
order to support decisions. Managers typically gather this data
manually, which takes time and effort. Jenkins and other CI tools
give you the ability to check trends and make choices frequently.
• Real-time metrics for code quality and build status: With a CI
system, integrations happen frequently, making it possible to spot
 trends in build and overall quality. Jenkins and other CI tools give
the team member metrics on the build's health. Since all build,
packaging, and deployment tasks are automated and monitored by a
continuous integration tool, statistics about the status of each
individual task can be produced. The build failure rate, build success
rate, number of builds, the source of the build, and other metrics can
be included in these metrics. Project managers and the team can use
all these trends to make sure that the project is moving forward at
the proper pace and direction. Additionally, continuous integration
includes static code analysis, which again provides a static report of
the code quality for every build. Code style, complexity, length, and
dependency are a few metrics that are very important.
• Rapid problem detection: Quick issue detection is the primary
benefit of a carefully designed CI system. Any merge or integration
issue is identified early. As soon as a build fails, notifications can be
sent via the CI system.
• Invest more time in feature addition: Building, releasing, and
deploying were once handled by development teams. After that, it
became common practice to assign build, release, and deployment
tasks to a separate team. That was once again insufficient because
there were communication problems between the development team
and the release team for this model. However, all build, release, and
deployment work are automated when using CI. As a result, the
development team only needs to worry about creating features. Even
the testing that has been completed is typically automated.
• Quick development: Technically speaking, CI makes teams more
productive. This is because CI is based on agile principles. When
building, testing, and integrating their code, projects that use CI
adopt an automated and continuous methodology. As a result,
development happens more quickly. Since everything is automated,
programmers only spend a small amount of time building,
packaging, integrating, and deploying their code. This makes it
easier for teams to collaborate even though they are spread out
geographically. Work can be done in large teams if a strong software
 configuration management process is in place. Agile development
can be further improved by Test Driven Development (TDD),
which increases its effectiveness.

Critical concepts of continuous integration

Only a properly executed CI process can be considered effective. By
streamlining the software development process, CI ensures no disconnected
stages. It achieves this by incorporating all the development phases—
integration, testing, verification, and deployment—into each segment of
development.
The following are the steps for CI:
1. Developer checks the code into the Version control tool’s staging
repository.
2. The version control system (VCS) notifies the CI server that a
commit has occurred. Or, the CI server polls the repository
periodically looking for commits.
3. The CI server starts the build process on a build server.
4. The code containing the latest commit is checked out of the
repository into a local workspace on the build server.
5. The changed code’s quality is tested, and the code is built.
6. Test results and build results are reported back to the CI server.
7. The CI server sets the final result of the build as Pass or Fail.
8. If the build is successful, then the committed change may proceed
through the development cycle - transferred to the real repository or
merged to the main development stream. If the build failed, the
committed changes are stopped from proceeding until those issues
are resolved by the developer.
9. The CI server notifies everyone who have registered interest in the
build. They can then log into the CI server to view the status plus
any additional information.
Refer to the following figure showcasing the steps for CI:

Figure 5.1: Continuous integration

Key practices of continuous integration

Just because a tool exists does not mean CI has been accomplished. The
configuration of the CI tool requires a significant amount of time. To
achieve CI, a tool like Jenkins collaborates with a variety of other tools. Let
us examine some of the most effective CI techniques:
• Developers should use the private workspace: Working in a
private workspace is always advised in a CI environment. Isolation
is the straightforward reason for this. On one's private branch, or
simply put, with one's private copy of the code, anyone can do
anything. The private copy is separated from the changes made to
the mainline branch once it has been branched. Developers are given
the freedom to experiment and try new things with their code in this
way.
• Rebase regularly from the mainline: Rebasing is simply updating
your private branch with the Integration branch's most recent
version. While working on a private repository or branch
undoubtedly has benefits, it also raises the risk of numerous merge
issues. Each developer contributing to a private clone of the primary
repository in a software development project with 10 to 20
 developers over time completely alters the primary repository's
appearance. Such circumstances are uncommon in a setting where
code is frequently merged and rebased. The benefit of using CI is
that we frequently and continuously integrate it.
• Frequent check-in: Regular check-ins on one's working branch
should go hand in hand with frequent rebases. It is risky to check in
more frequently than once per week. There could be merge
problems with the one week's worth of unchecked-in code. The
solution to these can be time-consuming. Conflicts are quickly
identified and can be resolved immediately by committing or
merging once per day.
• Regular build: Every commit or merge must be built using CI tools
in order to assess the effect of the change on the system. Continually
checking the integration branch for updates will help you
accomplish this. If changes are discovered, build and test them.
Share the outcomes with the group as soon as possible. Additionally,
builds can run nightly. The idea is to get instant feedback on the
changes they have made.
• Automate as much of the testing as you can: While a continuous
build can provide instant feedback on build failures, Continuous
testing, on the other hand, can assist in quickly determining whether
a build is ready to go into production. As many test cases as possible
should be included, but again, doing so will only make the CI
system more complex. The tests that most closely represent real-
world scenarios are the ones that are most challenging to automate.
Because there is so much scripting involved, keeping it up to date
costs more money. However, the more automated testing we have,
the more effectively and quickly we know the results.
• Automate the deployment: Deployment operations are handled by
a separate team in many organizations. Here is how it works. After
successfully completing a build, the developer submits a ticket or
emails requesting a deployment in the appropriate testing
environment. The testing team then confirms with the deployment
team that the environment is unoccupied. In other words, is it
 possible to stop testing for a few hours to allow for a deployment? A
specific time slot is chosen after some discussion, and the package is
then deployed. The majority of the deployment is done manually,
and there are numerous manual checks that take time. The developer
has to wait a whole day for a small piece of code to be sent to the
testing environment. And in some cases, it takes a whole day for the
code to reach the testing area if the manual deployment fails due to
human error or technical problems. For a developer, this is a painful
situation. The deployment procedure can be carefully automated,
though, to prevent this. When a developer attempts to check in code,
it first undergoes an automated compilation check, followed by an
automated code analysis, and then it checked in to the integration
branch. Once more, the most recent code from the Integration
branch is selected and then built. When a build is successful, the
code is automatically packaged and deployed in the testing
environment.
• Immediate notifications: They claim that without feedback,
communication is insufficient. Imagine a CI system that has
everything you need, including an automated build and deployment
solution, a cutting-edge automated testing platform, and a sound
branching strategy. It does not, however, have a notification system
that instantly emails or texts users about the status of a build. What
if the developers are not aware that a nightly build is failing? What
if you check in your code and leave right away, skipping the
automated build and deployment process? You discover the very
next day that the build was unsuccessful because of a simple
problem that appeared just 10 minutes after you left the office. You
could have resolved the problem if, by chance, you had received
notification via an SMS that popped up on your mobile phone.
Therefore, immediate notifications are crucial. It is present in every
CI tool, including Jenkins. Notifications of build failures,
deployment failures, and test results are beneficial.
In the upcoming chapters, we will see how Jenkins and its various options
can be used to accomplish this and make life easier.
Declarative build script
Your first step in implementing a CI solution should be to script your build,
regardless of the stage of your project—whether it is Greenfield, legacy, a
small individual library, or a large multi-module project. The frustration of
buggy build permutations caused by bad dependency management,
forgetting to include necessary resources when creating the distributable
package, or accidentally skipping build steps, among other pitfalls, can be
avoided by having a consistent, repeatable process that you can automate.
By scripting your build, you will save a huge amount of time. Your project's
build lifecycle, which consists of all the discrete steps needed to build it,
can easily become more difficult as time goes on, especially if you continue
to add modules, tests, and dependencies while also consuming an increasing
number of resources and dependencies. Depending on the environment
where your project will be deployed, you might also need to build it
differently. For instance, it may be necessary to enable debugging features
in a development or QA environment but to disable debugging in a build
that will be released to the public and to stop test classes from being
included in the distributable package. Manually carrying out every
necessary step needed to build a Java project, including taking into account
configuration variations per environment, is a breeding ground for human
error. You will understand the value of a build script the first time you skip
a step like creating an updated dependency and must repeat a build of a
sizable multimodule project to fix your error.
Make sure to use a declarative rather than an imperative approach when
scripting your build, no matter what tool or framework you select. Here is a
brief explanation of what these terms mean:
• Imperative: Setting up a precise process with implementation
information.
• Declarative: Defining an action without providing implementation
information. In other words. Keep your build script focused on what
you need to do rather than how to do it.
By promoting reuse on other projects or modules, this will keep your script
understandable, maintainable, testable, and scalable. To do this, you might
need to create or follow a well-established convention, or you might need to
create plug-ins or other external code that is referenced from your build
script and provides the implementation details. Some build tools are more
likely than others to encourage a declarative approach. This typically has a
trade-off between flexibility and adhering to convention.
The Java ecosystem has several well-established build tools available. This
section lists some of the most popular build tools and frameworks in the
Java ecosystem along with the default features they offer.
To get the most out of your build script, it is crucial to first map out your
build process and identify the requirements. The bare minimum
requirements for creating a Java project are as follows:
• Java version: The Java version needed to build the project.
• Source directory path: The directory containing the project's entire
source code.
• Destination directory path: The directory that should contain
compiled class files.
• Details of needed dependencies: The metadata required to locate
and collect any dependencies needed by your project.
With this knowledge, you should be able to carry out a basic build
procedure by doing the following:
• Gather all necessary dependencies.
• Compile the code.
• Run tests.
• Package your application.
An example is the most effective way to demonstrate how to incorporate
your build process into a build script. The examples below show how to
script the basic build procedure for a simple Hello World Java application
using three of the most popular build tools. These examples in no way
cover all of the features offered by these tools. They are merely intended to
serve as a basic course to assist you in either starting to understand your
current build script or writing your first build script in order to gain the
benefits of a complete CI solution.
Consider the actual steps your project requires to finish a build when
evaluating a build tool. One build tool might be better suited than another
for this task depending on your project, and you might need to script
additional steps not shown here. It's crucial that the tool you select enables
you to quickly define and speed up the build process that your project
needs, as opposed to arbitrarily forcing you to change your workflow to
satisfy the tool's specifications. Having said that, when you become familiar
with a tool's capabilities, consider your current workflow and think about
any adjustments that might be advantageous for your team.

Build with Apache Ant

Apache Ant is an open-source project released under an Apache License by
the Apache Software Foundation. According to the Apache Ant
documentation, the name is an acronym for another neat tool, which was
initially used to build Tomcat and came as an in-built product of the Tomcat
distribution kite. It was created by James Duncan Davidson on July 2000
for the purpose of building Tomcat.
Apache Ant is a build tool written in Java that provides a way to describe a
build process as declarative steps within an XML file. Although Ant has
heavy competition today, it is still an active project and widely used often in
combination with other tools. Now, let us discuss some key Ant
terminologies.

Key Ant terminology

You will encounter the following terms when working with Apache Ant:
• Ant task: A brief task, like copying a file or deleting a directory.
Ant tasks actually map to Java objects, which contain the task's
implementation information. Ant offers a large number of built-in
tasks as well as the option to create custom tasks.
• Ant target: Ant targets are collections of Ant tasks. An Ant target is
directly called by Ant. For instance, you would issue the command
ant compile for a target named compile. Ant targets can be set up to
depend on one another in order to manage the execution sequence.
There are some Ant build files that can get quite big. To get a list of
available targets, enter the following command in the same directory
as [Link]:
ant -projecthelp
• Ant build file: An XML file that is used to set up every Ant task
and target use by a project. This file can be found at the root of the
project directory by default and is called [Link]. Example 5.1
shows an Ant build script:

<project name="MyProject" default="package" basedir=".">❶

<!-- Define property values -->❷

<property name="[Link]" value="src/main/java" />
<property name="[Link]" value="target" />
<property name="[Link]" value="${[Link]}/classes" />
<property name="[Link]" value="src/test/java" />
<property name="[Link]" value="${[Link]}/test-classes"
/>
<property name="[Link]" value="lib" />

<!-- Path to include jar files -->❸

<path id="classpath">
 <fileset dir="${[Link]}">
<include name="**/*.jar" />
</fileset>
</path>
<!-- Clean target -->
<target name="clean">
<delete dir="${[Link]}" />
</target>

<!-- Compile target -->❹

<target name="compile" depends="clean">
<mkdir dir="${[Link]}" />
<javac srcdir="${[Link]}" destdir="${[Link]}" />
</target>
<!-- Compile-test target -->
<target name="compile-test" depends="compile">
<mkdir dir="${[Link]}" />
<javac srcdir="${[Link]}" destdir="${[Link]}">
<classpath refid="classpath"/>
</javac>
</target>

<!-- Test target -->❺

<target name="test" depends="compile-test">
<junit printsummary="on" haltonfailure="no">
 <classpath refid="classpath"/>
<batchtest fork="yes" todir="${[Link]}">
<fileset dir="${[Link]}">
<include name="**/*[Link]"/>
</fileset>
</batchtest>
</junit>
</target>

<!-- Package target -->❻

<target name="package" depends="test">
<mkdir dir=”${[Link]}/jar” />
<jar destfile=”${[Link]}/jar/[Link]” basedir=”${[Link]
r}” />
</target>
</project>
Example 5.1: Ant build script ([Link])

❶ When Ant is called without a target, the value of the default attribute of
the project can be set to the name of a default target to execute. The
package target for this project will be run by the command ant without any
arguments.

❷ The rest of the build script may use property elements multiple times
because they are hardcoded, immutable values. It is easier to read and
maintain code when they are used.

❸ We have decided to manage the location of necessary dependencies for

this project using this path element. Here, the junit and hamcrest-core JARs
are both manually added to the specified directory. By using this method,
dependencies are implied to be checked into source control alongside the
project. Although it was easy to do in this example, it is not a good idea to
follow this procedure.

❹ The compilation of the source code (with latest Java version) and
placement of the resulting class files in the designated location are the
responsibilities of the compile target. Because this target depends on the
clean target, the clean target will be executed first to make sure that the
compiled class files are current and not outdated.

❺ The test target sets up the JUnit Ant task, which executes all of the unit
tests that are available and prints the results to the screen.

❻ A final JAR file will be assembled and put in the designated location by
the package target.
Our Java project will be taken, compiled, put through unit tests, and then
assembled into a JAR file for us by running the single-line command ant
package. Ant meets our requirement of scripting a simple build because it is
adaptable, feature-rich, and flexible. The project's build lifecycle is easily
documented using the XML configuration file. Ant does not effectively
manage dependencies on its own. However, programs like Apache Ivy have
been created to give Ant access to these features.

Build with Apache Maven

Maven, a Yiddish word meaning accumulator of knowledge, began as an
attempt to simplify the build processes in the Jakarta Turbine project. Like
Apache Ant, Maven is also an open-source project of the Apache Software
Foundation. Its first official release was in 2004.
Like Apache Ant, Maven manages and describes Java projects using an
XML file called a project object model (POM). A unique identifier for the
project, the necessary compiler version, configuration property values, and
metadata on all necessary dependencies and their versions are all recorded
in this document. Maven's ability to manage dependencies and use
repositories to share dependencies with other projects is one of its most
potent features.
Maven heavily relies on convention to provide a standardized approach to
managing and documenting a project that can easily scale across all Maven-
based projects. On the file system, a project is expected to be organized in a
particular way. Custom implementations require custom plug-ins to
maintain the script's declarative nature. Maven can be heavily customized to
override expected defaults, but if you adhere to the expected project
structure, it works out of the box with little configuration.

Key Maven terminology

When working with Maven, you may come across the following terms:
• Lifecycle phase: A discrete step in a project’s build lifecycle. The
Maven build follows a specific lifecycle to deploy and distribute the
target project.
There are three built-in lifecycles:
• default: The main lifecycle, as it's responsible for project
deployment.
• clean: To clean the project and remove all files generated by
the previous build.
• site: To create the project's site documentation.
• Each lifecycle consists of a sequence of phases. Maven defines a list
of default phases that are executed sequentially during a build. The
default phases are validate, compile, test, package, verify, install,
and deploy. Invoking Maven with a lifecycle phase will execute all
of the lifecycle phases in order, up to and including the given
lifecycle phase.
• Maven goal: Handles the details of how a lifecycle phase is
executed. Multiple lifecycle phases can be configured to be
associated with a goal.
• Maven plug-in: A Maven plugin is a collection of goals, but they
aren't all required to belong to the same phase. Plug-ins offer goals
for the lifecycle phase they are bound to, and these goals are then
carried out.
• POM file: The configuration for each Maven lifecycle phase, goal,
and plug-in needed for the project's build lifecycle is contained in
the Maven Project Object Model, or POM, which is implemented as
an XML configuration file. The project's root contains a file with the
name [Link].
Example 5.2 is a simple POM file configured for Java 17
environment using Maven 3.11.0:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="[Link]
xmlns:xsi="[Link]
xsi:schemaLocation="[Link] [Link]
[Link]/xsd/[Link]">
<modelVersion>4.0.0</modelVersion>

<!-- Define project coordinates -->❶

<groupId>[Link]</groupId>
<artifactId>my-project</artifactId>
<version>1.0.0</version>
<name>My Project</name>

<!-- Define properties -->❷

<properties>
<[Link]>UTF-8</[Link]
ding>
<[Link]>17</[Link]>
 <[Link]>5.8.0</[Link]>
</properties>

<!-- Define dependencies -->❸

<dependencies>
<dependency>
<groupId>[Link]</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>${[Link]}</version>
<scope>test</scope>
</dependency>
<!-- Add any other dependencies your project needs -->
</dependencies>

<!-- Define build configuration -->❹

<build>

<plugins>❺
<!-- Maven Compiler Plugin -->
<plugin>
<groupId>[Link]</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.11.0</version>
<configuration>
<source>${[Link]}</source>
<target>${[Link]}</target>
 </configuration>
</plugin>
<!-- Surefire Plugin for running tests -->
<plugin>
<groupId>[Link]</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<argLine>--enable-preview</argLine>
</configuration>
</plugin>
</plugins>
</build>
</project>
Example 5.2: Maven POM file ([Link]) using Maven

❶ Every project is uniquely identified by its configured groupId,

artifactId, and version.

❷ Properties are hardcoded values that might be used in several different

places throughout the POM file. They may be built-in or customized
properties that are used by plug-ins or goals.

❸ Every direct dependency of the project is listed in the dependencies

block. To run the unit tests for this project, JUnit is required, so this section
specifies the dependency on JUnit. Although JUnit itself depends on
hamcrest-core, Maven is wise enough to know this and doesn't need to
include it here. Maven will pull these dependencies from Maven Central.
❹ In the build block, plug-ins are configured. This block is not necessary
unless there is configuration you want to override.

❺ There are default plug-in bindings for each phase of the lifecycle, but in
this case, we wanted to set the maven-compiler-plugin to use Java 17 rather
than the default. [Link] is a property in the properties
block those controls this for the plug-in. Although this configuration could
have been placed in the plugins block, it makes more sense to move it to the
properties block at the top of the file for better visibility. When using older
versions of Java, this property takes the place of [Link] and
[Link].
Spending any time with Maven will make it clear that it is much more than
just a build tool and that it is packed with useful features. Its potential
complexity can be overwhelming for a Maven newbie. Additionally, when
the customization is minor, creating a new Maven plug-in is challenging.

Build with Gradle

Gradle is an Apache 2.0 licensed open-source build automation tool that
focuses heavily on performance. In contrast to Apache Ant and Apache
Maven, Gradle was only introduced with the release of version 1.0 in 2012.
The Gradle build script does not use XML, which is one of the most
significant differences between Gradle, Maven, and Ant. Instead, a Kotlin
DSL or Groovy can be used to write Gradle build scripts. Gradle follows
convention just like Maven, but it is more flexible. The Gradle
documentation highlights the tool's adaptability and offers guidelines for
quickly customizing your build. Now let’s look at Key gradle
terminologies.

Key gradle terminology

You will encounter the following terms when working with Gradle:
• Domain-specific language: The Domain-specific language or
DSL used by Gradle scripts is specific to Gradle. You can use either
the Groovy or the Kotlin language features to write a Gradle script
using the Gradle DSL. The Gradle DSL is documented in the Gradle
Build Language Reference.
• Gradle task: A distinct stage in your build process that can include
the execution of work units, such as copying files, as well as the
inputs and outputs that are used in the implementation. To control
the execution sequence, tasks can specify dependencies on one
another. Multiple tasks will be included in your project build, which
a Gradle build will configure and then execute in the proper
sequence.
• Gradle lifecycle tasks: Common tasks provided by Gradle’s Base
plug-in, including clean, check, assemble, and build. The Base plug-
in can be used by other plug-ins to gain access to these tasks.
• Gradle plug-in: A collection of Gradle tasks and the mechanism for
adding extensions to existing functionality, features, conventions,
configuration, and other customizations to your build.
• Gradle build phase: Gradle build phases are not to be confused with
Maven phases. Initialization, Configuration, and Execution are the
three fixed build phases that a Gradle build will go through.
In Example 5.3, we will see a simple Gradle build script:

plugins { ❶
id 'java'
id 'maven-publish'
}
group '[Link]'
version '1.0.0'
description = 'My Project Description'
sourceCompatibility = JavaVersion.VERSION_17 ❷

repositories { ❸
mavenCentral()
mavenLocal()
}

dependencies { ❹
testImplementation '[Link]:junit-jupiter-api:5.8.0'
// Add any other dependencies your project needs
}
test {
useJUnitPlatform()
}
publishing {
publications {
maven(MavenPublication) {
groupId = '[Link]'
artifactId = 'my-project'
version = '1.0.0'
description = 'My Project Description'
from [Link]
}
}
repositories {
 maven {
url = uri('[Link] + "${buildDir}/repo")
}
mavenLocal()
}
}

[Link](JavaCompile) { ❺
[Link] = 'UTF-8'
}
Example 5.3: Gradle build script ([Link])

❶ To use a Gradle plug-in, add its plug-in ID to the plugins block. A

Gradle Core plug-in called the "java plug-in" offers Java projects
functionality for compilation, testing, packaging, and other things.

❷ The java plug-in offers the sourceCompatibility configuration setting,

which maps to the source javac option. TargetCompatibility is another
configuration option. Its default value is the value of sourceCompatibility,
so there was no need to add it to the build script.

❸ The repositories block offers repositories for dependencies. These

settings deal with dependencies.

❹ Maven and Gradle both handle dependencies in the same way. Our unit
tests need the JUnit dependency, so it is listed in the dependencies block.

❺ We can add explicit encoding for the Java compiler thanks to Gradle's
flexibility. JavaCompile is the name of a task offered by the Java plug-in
that is of the JavaCompiler type. The encoding property for this compile
task is set by this code block.
By using the single command gradle build, this Gradle build script allows
us to compile, run tests, and assemble a JAR file for our project. Since
Gradle builds are based on well-established conventions, build scripts
contain only the information required to differentiate the build, keeping
them concise and maintainable. This simple script illustrates how flexible
and powerful Gradle can be, particularly for Java projects with more
complex build processes. In that case, making the initial investment in
understanding the Gradle DSL for customization is time well spent.
These three Java project-building tools have their advantages and
disadvantages. Based on your project's needs, your team's experience, and
the level of flexibility required, select a tool. The process of creating a Java
project is repetitive, involves many steps, is prone to human error, and is
ideally suited for automation. Reducing the build process for your project to
a single command speeds up development tasks in a local development
environment, reduces ramp-up time for new developers, and paves the way
for build automation, a pivotal component of an effective CI solution.

Continuous build
Failure of a build is the most apparent indication that code integration was
unsuccessful. As a result, a project should be developed frequently in order
to identify and address any problems as soon as possible. In fact, it should
be expected that every change made to the mainline codebase will result in
a build that compiles successfully and passes all unit tests.
An example of a test-driven development workflow for developers is as
follows:
1. Check out the most recent code to a local workspace from source
control.
2. Build and run all tests for the project to ensure a clean start. Build
script can be used for this.
3. Write the code and associated unit tests for the new feature or bug
fix.
 4. Run the new unit tests to make sure the new unit tests pass.
5. Build and run all unit tests for the project to ensure that the new
code doesn’t result in negative side effects when integrated with the
existing code. (Again, use a build script for this.)
6. Commit the new code along with the new tests to the codebase.
This process aims to avoid problems with integrating code before it leaves
your local development workspace, such as the introduction of bugs or a
loss of functionality. However, issues that will cause pain in the future can
emerge during this workflow. Human nature will play a role in some of
them, but others will arise because it is nearly impossible to foresee every
potential incompatibility brought about by concurrent development, no
matter how much effort is put into advance planning. In order to improve
your efficiency and productivity as a developer, this section explains how
an automated CI implementation helps to mitigate the issues that may occur
during this workflow.
Successfully build a project in one's own local environment is not sufficient
for a developer. You should not solely rely on developers adhering to the
agreed process and only committing code changes after all tests have
passed. The simplest reason is that a developer might not have the latest
changes from the mainline (a scenario that is even more likely when
numerous developers are working on the same codebase). This might lead
to an incompatibility that isn't noticed until the code has been merged.
Sometimes, issues with tests do not become apparent until someone else
tries to build or run the tests in their own local development environment.
For example, someone forgets to commit a newly created resource or new
file to the codebase. This simply means that the next developer who
combines these changes will have to put up with the annoyance of having
the build or tests fail immediately. Another issue is that Code is written in
such a way that it only runs in a particular environment or on a particular
OS.
Even in the best circumstances, we all have bad days, and sometimes, these
issues pass us by. However, strategies can be put in place to help mitigate
integration issues like this rather than allowing broken builds to spread
throughout the team like a virus. The most common is using automatic
build servers, or CI servers. The development teams using these servers are
responsible for performing full builds, including running tests and reporting
the result of the build after code changes have been committed.
Jenkins, TeamCity, Bamboo, GitHub actions, Azure DevOps, and GitLab
are a few well-known CI servers that you may be familiar with. Although
some have more features and capabilities than others, some have more
features and capabilities than others, but the main goal is to establish a
referee for the code changes entering the shared repository by frequently
building and reporting when a problem arises. The best way to make sure
that builds are happening regularly and identify any integration issues early
is to use a CI server for the purpose of running builds automatically.

Test automation
In addition to running individual tests during development, typically within
an IDE, A developer should have a quick method of running the entire suite
of automated tests before checking in new code to the codebase.
The minimal build process outlined in the section, Declarative Build Script
includes a step for automatically running unit tests. This unit test step is
also present in every build script. This is not by chance. In fact, this part of
your build needs a good amount of time and attention because it is
absolutely necessary for a healthy CI solution. Having the ability to identify
integration issues as early in the development process as possible is one of
the main goals of CI.
The expectation that unit tests will discover every issue is unrealistic.
However, one of the best proactive strategies for finding the most obvious
issues early on is writing a strong set of unit tests. Unit tests are a crucial
component of the development cycle because they can be executed even
before the initial phases of formal quality assurance. They are the initial set
of safety precautions you can take to guarantee that your software will
operate properly in a production environment.
We will assume that you are conscious of the importance of unit tests and
use a framework like JUnit or TestNG to make it easy to run them
automatically. In order to write and maintain unit tests going forward, you
should sit down with your development team and plan your approach.
In addition to unit tests, it is important to run integration tests as a part of
your CI process. While unit tests validate individual components in
isolation, integration tests ensure that different system parts work together
as expected. Tools like Cucumber, Selenium, and Postman are commonly
used for integration testing. Cucumber supports behavior-driven
development (BDD) and helps verify interactions between components
based on business scenarios, while Selenium is useful for automating
browser-based tests, ensuring that your web application integrates well
across various systems. Postman, on the other hand, is excellent for API
integration testing, validating that different services communicate
effectively. By incorporating integration testing with these tools, you can
detect issues in the interactions between modules early in the development
process. This step complements your unit testing strategy, helping to catch
integration issues before they affect the stability of the software in
production.

Monitor and maintain tests

The most recent code can be easily checked out to your local development
environment. It is also simple to compile and run every unit test. The time it
takes to perform a full build and run all the tests will increase as you add
more modules and your project becomes more complex. The longer your
development cycle, the more likely other code changes have been made and
pushed to the mainline before yours.
You would need to review the most recent changes and rerun each test in
order to avoid a potential break, which is not a very efficient process. The
desire to commit code to the mainline before it is changed out from under
them can drive developers to take shortcuts and skip running tests out of
frustration.
Test maintenance requires time, and this time should be regularly built into
the development schedule. Tests need to be improved and modified over
time, just like the rest of your codebase. They need to be removed
immediately when they become obsolete. They ought to be repaired when
they break.
There are times when tests are not run because it is determined that they
take too much time. Define your acceptable threshold by regularly tracking
the length of test runs on your CI server. Consider stopping to review your
tests as your project continues to expand and amount of time for builds
exceeds your acceptable threshold. Look for tests that are redundant,
outdated, or that can be run concurrently. Every second counts when you
consider how frequently you expect your build server to run (possibly after
each code change).

Building pipeline with Jenkins

A pipeline is a sequence of tasks or events that can be executed. Visualizing
a series of stages is the simplest way to understand a pipeline. Numerous
tasks in the CI/CD pipeline are automated by the Jenkins pipeline, which
also makes them reliable, efficient, repeatable, and of a high quality.
Additional features like audit trails, code review, access control, and a
sound approval and promotion process by numerous project members have
made the iterative development process code easier. It is simple to manage
multiple jobs from a single project. There are various advantages of using
the Jenkins pipeline, a few of which are as follows:
• It is possible to review the code in the pipeline.
 • Make many pipelines automatically for all the branches with a
single Jenkins file. With a single Jenkins file, numerous pipelines
are created automatically for all the branches.
• The Jenkins pipeline can be reviewed. One can review the Jenkins
pipeline.
• We can use the Web User interface or Jenkins file directly.
Alternatively, we can use a Jenkins file directly or the Web User
interface.
Here, you should see two familiar concepts, Stage and Step:
• Stage: A block that contains a series of steps. A stage block can be
named anything; it is used to visualize the pipeline process, a block
that has several steps in it. Any name can be used for a stage block,
which is used to represent the pipeline process.
• Step: A task that says what to do. Steps are defined inside a stage
block.
The following (Example 5.4) shows a simple Jenkins pipeline including the
code checkout, build, test, and code publish artefact steps:
pipeline {
agent any
stages {

stage('Checkout') { ❶
steps {
git branch: 'main', url: '[Link]
}
}

stage('Build') { ❷
steps {
 sh 'mvn clean package'
}
}

stage('Test') { ❸
steps {
sh 'mvn test'
}
post {
always {
junit 'target/Test-reports/**/*.xml'
}
}
}

stage('Publish Artifact') { ❹
steps {
sh 'mvn deploy'
}
}
}
}
Example 5.4: Jenkins Pipeline Example (Jenkinsfile)

In this script, the pipeline has four stages as follows:

❶ Checkout: It checks out the code from the specified Git repository.
❷ Build: It runs the Maven clean package command to build the project
and package the artifact.

❸ Test: It runs the Maven test command to execute the JUnit tests. The
JUnit step is then used to publish the test results generated by Surefire.

❹ Publish Artifact: It runs the Maven deploy command to publish the

artifact to a Maven repository.
You can modify the script as per your requirements, such as changing the
Git repository URL, branch, or Maven commands.

Conclusion
CI was emphasized in this chapter as a crucial practice for a development
team. Over time, tools have emerged that have improved our ability to
construct software projects. The ability to run tests and trigger builds
automatically has allowed developers to focus more intently on their coding
and find mistakes earlier in the development cycle.
We began by delving into various aspects of CI implementation. We
discussed its significance within development teams, discussed its critical
concepts, and the key practices that form the foundation of a successful CI
process.
The discussion then shifted to the practical aspects, focusing on
implementing declarative build scripts using tools like Ant, Maven, and
Gradle. This highlighted their role in simplifying and standardizing the
build process, fostering efficiency. We also explored Continuous Build
processes, emphasizing their continuous and iterative nature, contributing to
the overall agility of the development lifecycle.
Lastly, we explored the pivotal role of test automation, emphasizing its
contribution to reliability and efficiency. We underscored the importance of
ongoing test monitoring and maintenance for sustaining a resilient CI
environment. The chapter concludes by providing practical insights into
constructing a building pipeline with Jenkins. This offers an actionable
strategy to implement and optimize CI workflows for enhanced software
development practices.
In the next chapter, we will discuss package management and how the
presence of metadata and dependencies can enhance the health and stability
of a pipeline.

Join our book’s Discord space

Join the book’s Discord Workspace for Latest updates, Offers, Tech
happenings around the world, New Release and Sessions with the Authors:
[Link]
 CHAPTER 6
Package Management

Introduction
As you read this sentence, a line of code is being written somewhere in the
world. This line of code will eventually be included in an artifact that will
act as the building block for one or more enterprise products developed
internally by a company or shared via a public repository, most notably
Maven Central for Java and Kotlin libraries.
Today, there are more libraries, binaries, and artifacts readily available than
ever before, and this collection will keep expanding as programmers from
all over the world continue to create the latest products and services. With
an ever-growing number of dependencies forming a complex web of
connectedness, handling and managing these artifacts now requires more
work than it did in the past. It is easy to fall into the trap of using the
incorrect version of an artifact, which leads to confusion, broken builds, and
ultimately delays the release of carefully planned projects.
For developers, it is more crucial than ever to understand not only the
features and functionality of the source code that they are currently working
with, but also how their projects are packaged and how the building blocks
are assembled to create the final product. It is essential to have a thorough
understanding of the build process itself and how our automated build tools
work in order to avoid delays, hours of pointless troubleshooting, and the
possibility of a significant number of bugs making their way into
production.
Access to a riches of third-party resources that address common coding
issues can speed up the development of our projects but also increases the
possibility of errant or unexpected behavior. It will be easier to troubleshoot
problems if you know how and where these components are brought into
projects. We will be better able to prioritize and make better decisions when
it comes to bug fixes and feature development, and we will also be able to
help pave the way for the release to production if we make sure we are
responsible managers of the artifacts we create internally. A developer now
needs to understand the complexities of the package management in
addition to the semantics of the code in front of them.

Structure
In this chapter, we will discuss the following topics:
• Understanding metadata
• Understanding artifact
• Package management
• Dependency management
• Dependency management in containers
• Artifact publication

Objectives
By the end of this chapter, you will understand the package management
and what are the things we do for packaging a software product. This
chapter will explain what metadata is and its importance, how to create and
capture metadata using Java properties files and JAR's Manifest.
Additionally, it aims to discuss artifact generation, package management,
dependency management in Apache Maven, Gradle, and containers,
detailing how dependencies are managed and resolved to ensure project
efficiency.
We will also explore artifact publication, highlighting its benefits and best
practices. We will discuss the methods of publishing artifacts to repositories
like Maven Local, Maven Central, Sonatype Nexus Repository, and JFrog
Artifactory.

Understanding metadata
Until recently, software developers saw the creation of an artifact as the
culmination of challenging, occasionally epic, efforts. Sometimes, in order
to meet deadlines, steps had to be hurriedly completed. Since then, the
industry’s demands have evolved to include multimodule packages,
diversified environments, tailored artifacts, exploding codebases, and
repositories. In modern times, building an artifact is only one stage of a
bigger business cycle.
The best innovations come from trial and error, as successful leaders are
aware of. They have thus incorporated testing, experimentation, and failure
into both their daily lives and the operations of their business.
One way to innovate, scale more quickly, release more products, enhance
the usability or quality of applications or products, and introduce new
features is by using the most fundamental, century-old technique of
comparing two versions of something to determine which performs better.
Comparative testing is done by many businesses on each new feature. The
company is able to continuously enhance the user experience by
benchmarking various versions against one another and monitoring
customer feedback.
How do we deliver and deploy various multiple versions of software made
up of various artifacts? How are bottlenecks discovered? How can we be
certain that our progress is correct? How do we keep track of what is
helping us or hurting us? How can we continue to have reproducible results
but with enriched lineages? It is possible to find the answers to these
questions by gathering and studying relevant, clear, and specific data about
the inputs, outputs, and states of the workflows and artifacts. Thanks to
metadata, all of this is possible.
Data that offers details about other data is referred to as Metadata. The
format, structure, content, location, and context of a specific piece of data
are just a few examples of the various characteristics and qualities it
describes. In other words, it is a set of characteristics or attributes that apply
to a specific entity, in our case, artifacts and processes. In order to make
data easier to search for, retrieve, and use, metadata is used to organize,
manage, and comprehend the data. Metadata enables the identification of
correlations and causes as well as insights into the behavior and outcomes
of the organization. As a result, metadata can demonstrate whether an
organization is responsive to the objectives of its stakeholders.
Importance of metadata can be summarized as follows:
• Identification and description: Users can better understand the
purpose, relevance, and potential uses of data by using metadata to
identify and describe it.
• Searchability and discoverability: By including metadata, data can
be more easily found and searched. Metadata can provide keywords,
tags, or labels that help in order to categorize and classify data to
make it easier to find pertinent information.
• Contextual understanding: The source, date of creation, author,
and other pertinent details are all provided by the metadata, which
adds valuable context to the data. This background information
improves how the data are understood and interpreted.
• Data integrity and quality: Metadata can include information
about Data reliability, accuracy, and quality. It enables users to
assess whether the data is reliable and appropriate for their specific
needs.
 • Integration and interoperability: Metadata standards make it
easier to exchange and integrate data between various platforms and
systems. The sharing and interoperability of data among various
applications and organizations are made possible by consistent
metadata formats and structures.
Now, let us take a look at key attributes of metadata:

Key attributes of insightful metadata

Insightful metadata plays a crucial role in organizing and providing
meaningful information about various types of data. Here are key attributes
that contribute to insightful metadata:
• Descriptive title: A title for metadata should be concise and
accurately reflect the information's content or goal. Users can gain a
high-level understanding of the data's meaning from the title.
• Description: A thorough description gives the data more context
and information. Users should be able to assess the data's suitability
for their needs by understanding the data's purpose, scope, and
relevance.
• Tags and keywords: Including relevant tags and keywords
improves discoverability and searchability of the data. These
descriptive terms, which may include particular terms, categories, or
concepts associated with the data, should be indicative of the
content.
• Source or origin: Identifying the source or origin of the data is
essential for the purpose of establishing credibility and
comprehending the provenance of the data. It helps users in
determining the data's dependability and credibility.
• Format and structure: Data formats and structures, such as file
types, encodings, schemas, and data models, should be specified in
the metadata. Understanding how to effectively interpret and use the
data requires this information.
 • Data dictionary: Including a data dictionary that explains the
meaning and usage of specific data elements or fields can improve
understanding and facilitate data integration. Data consumers can
use it as a clear guide to correctly interpret the data.
• Date and versioning: Metadata should indicate the creation date of
the data, as well as any updates or new versions of the data. This
information helps users to keep track of updates, evaluate the data's
currency, and locate the most recent version.
• Access and usage rights: It is crucial to clearly define any access
restrictions, licensing terms, or usage rights related to the data. By
doing so, users are better able to comprehend how to access, share,
and utilize the data while remaining compliant with any applicable
legal or licensing requirements.
• Relationship to other data: Metadata can provide information on
the relationships between different datasets, such as dependencies,
links, or integration points. This allows users to understand the
context and connections between related data resources.
• Documentation and examples: Comprehensive documentation,
along with relevant examples, improves the data's usability. It
should provide instructions, guidelines, or examples of code to help
users work with and analyze the data in an efficient manner.
By including these key attributes, metadata can offer insightful information
and context about the data, assisting users in finding, comprehending, and
making better use of the data for their individual needs. Now Let’s look at
the steps to create metadata.

Creating metadata
To create metadata, consider the following steps:
1. Identify relevant attributes: Determine the essential characteristics
and properties that are important for describing and organizing your
data. Title, author, date, format, keywords, subject, and other
information can be included in these attributes.
 2. Choose metadata standards: Select metadata standards or schemas
that are appropriate for your data type and purpose. Dublin Core,
Metadata Object Description Schema (MODS), and [Link]
are examples of well-liked standards.
3. Define metadata elements: Based on the chosen standards, define
the specific metadata elements that need to be included. When using
Dublin Core, for instance, information like title, creator, subject,
description, date, and format may be included.
4. Capture and document metadata: Create a process for capturing
and documenting the metadata for your data. This may involve both
manual entry and automated extraction, or both. When capturing,
make sure your metadata collection is accurate and consistent.
5. Apply metadata to data: Associate the corresponding data and the
captured metadata. This can be achieved by either maintaining a
separate metadata repository or catalog or by embedding metadata
within the data file itself.
6. Maintain and update metadata: Review and update the metadata
frequently to reflect any data changes or updates. To ensure that
metadata remains accurate and useful, keep it up to date.
By following the preceding steps, you can create effective metadata that
enhances data organization, searchability, and understanding.

Capturing metadata
Once we know which metadata we need to capture, we need to figure out
how to gather it with our preferred build tool. Some key/value pairs can be
obtained directly from the environment, system settings, and command
flags that the JVM exposes as environment variables or system properties.
Additional properties may be exposed by the build tool itself, whether they
are defined as extra command-line arguments or as configuration elements
in the tool's configuration settings.
Let us assume for the time being that we need to capture the following
key/value pairs:
 • JDK information, such as version and vendor
• OS information such as name, arch, and version
• The build timestamp
• The current commit hash from SCM (assuming Git)
With Maven, these values can be captured by combining system properties
for the first two items and a third-party plug-in for the final two. There are
numerous options for plug-ins that offer integration with Git in both Maven
and Gradle. But let us choose git-commit-id-maven-plugin for Maven and
versioning for Gradle here.
Currently, Maven permits defining properties in a few different ways, most
frequently as key/value pairs inside the <properties> section located in the
[Link] build file. Each key's value is free text, but you can use a
shorthand notation to refer to System properties or a naming convention to
refer to environment variables. Consider the scenario where you want to
access the value of the [Link] key in the system properties. The
shorthand notation $ { } can be used to accomplish this, for example,
${[Link]}. Conversely, for an environment variable, you may use the
${[Link]} notation.
In this example, the key/value pairs are specified as properties within the
<properties> section of the [Link] file. The values for JDK version and
vendor, as well as OS name, arch, and version, are represented by Maven
properties (${property-name}).
The git-commit-id-maven-plugin is configured within the <build> section
of the [Link] file. It retrieves the current commit hash from the Git
repository and stores it in the [Link] property. The verbose property
is set to false to suppress unnecessary output, and the dateFormat property
defines the format of the build timestamp. The injectAllReactorProjects
property is set to true to include information from all modules within a
multi-module project.
Putting together the build properties and git-commit-id plug-in may result
in a [Link] similar to Example 6.1:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="[Link] xmlns:xsi="[Link]
[Link]/2001/XMLSchema-instance" xsi:schemaLocation="[Link]
[Link]/POM/4.0.0 [Link]
<modelVersion>4.0.0</modelVersion>
<groupId>[Link]</groupId>
<artifactId>my-project</artifactId>
<version>1.0.0</version>
<properties>
<!-- JDK Information -->
<[Link]>17</[Link]>
<[Link]>${[Link]}</[Link]>
<!-- OS Information -->
<[Link]>${[Link]}</[Link]>
<[Link]>${[Link]}</[Link]>
<[Link]>${[Link]}</[Link]>
<!-- Build Timestamp -->
<[Link]>${[Link]}</[Link]>
<!-- Git Commit Hash -->
<[Link]>${[Link]}</[Link]>
</properties>
<build>
<plugins>
<!-- Git Commit ID Plugin -->
<plugin>
<groupId>[Link]</groupId>
<artifactId>git-commit-id-plugin</artifactId>
<version>4.0.0</version>
<executions>
<execution>
<id>get-the-git-infos</id>
<goals>
<goal>revision</goal>
</goals>
</execution>
</executions>
<configuration>
<verbose>false</verbose>
<dateFormat>yyyy-MM-dd HH:mm:ss</dateFormat>
<injectAllReactorProjects>true</injectAllReactorProjects>
</configuration>
</plugin>
</plugins>
</build>
</project>
Example 6.1: Maven POM file ([Link]) with Maven capturing metadata
To use this [Link] file, make sure you have Maven installed on your
system. Navigate to the project's root directory containing the [Link] file
and execute Maven commands such as mvn clean install to build and
install the project.
After building the project, you can access the values of the properties in
your code using the [Link]("property-name") method. For
example, to retrieve the JDK vendor, you can use
[Link]("[Link]").
With this [Link] configuration, you can capture the JDK information, OS
information, build timestamp, and SCM commit hash within your Java
application. This information can be useful for logging, version tracking, or
providing diagnostic information.
By capturing JDK information, OS information, build timestamp, and SCM
commit hash in your [Link] file, you can provide valuable insights and
context about the artifact. This information helps users understand the
environment in which the artifact was built and provides important
versioning and tracking details.

Writing the metadata

Metadata may need to be recorded in multiple formats or files. The intended
consumers will determine the format to use. While some consumers may
only be able to read a specific format, others may be able to comprehend a
variety of formats. To learn more about the supported formats and options
for a specific consumer, refer to its documentation. You should also see if
integration with your preferred build tool is provided in the documentation.
You might discover that there is a plug-in for your build that makes it
simpler to record the necessary metadata. For demonstration purposes, we
will record the metadata by using two popular formats: a Java properties
file and the JAR’s manifest.
1. Java properties file: Create a file named [Link]
(Example 6.2) and add the following key/value pairs:
# [Link]
[Link]=17
[Link]=Windows
[Link]=x86_64
[Link]=2023-06-30 [Link]
[Link]=abc123
Example 6.2: [Link] file

In your Java code, you can read the properties file and access the
values using the [Link] class. Here's an example
(Example 6.3) :
import [Link];
import [Link];
import [Link];
public class MetadataExample {
public static void main(String[] args) {
Properties = new Properties();
try(FileInputStream fis = new FileInputStream
("[Link]")) {
[Link](fis);
} catch (IOException e) {
[Link]();
}
String jdkVersion = [Link]("[Link]");
String osName = [Link]("[Link]");
String osArch = [Link]("[Link]");
 String buildTimestamp = [Link]("[Link]");
String commitHash = [Link]("[Link]");
// Use the metadata values as needed
[Link]("JDK Version: " + jdkVersion);
[Link]("OS Name: " + osName);
[Link]("OS Arch: " + osArch);
[Link]("Build Timestamp: " + buildTimestamp);
[Link]("Commit Hash: " + commitHash);
}
}
Example 6.3: Reading Java properties file in Java code

2. JAR's Manifest: To add metadata to the JAR's manifest, you can

modify the [Link] file to include the necessary configurations.
Add the following plugin to the build section (Example 6.4):
<build>
<plugins>
<plugin>
<groupId>[Link]</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.11.0</version>
<configuration>
<archive>
<manifestEntries>
<[Link]>17</[Link]>
<[Link]>Windows</[Link]>
 <[Link]>x86_64</[Link]>
<[Link]>2023-06-30 [Link]</[Link]>
<[Link]>abc123</[Link]>
</manifestEntries>
</archive>
</configuration>
</plugin>
</plugins>
</build>
Example 6.4: Adding metadata to JAR’s manifest in [Link]

In this example, the maven-jar-plugin configures the JAR's

manifest entries. Keep in mind that you must set the Maven
Compiler Plugin version to be compatible with your current Java
version. In the above piece of code, the metadata key/value pairs are
specified within the manifestEntries section. After building the
project with Maven (mvn clean package), the JAR file will contain
the metadata in its manifest.
To access the manifest metadata programmatically, you can use the
[Link] class. Here is an example (Example 6.5):
import [Link];
import [Link];
import [Link];
import [Link];
public class MetadataExample {
public static void main(String[] args) {
try (JarFile = new JarFile("[Link]")) {
 Manifest = [Link]();
Attributes = [Link]();
String jdkVersion = [Link]("[Link]");
String osName = [Link]("[Link]");
String osArch = [Link]("[Link]");
String buildTimestamp = [Link]("[Link]
tamp");
String commitHash = [Link]("[Link]
h");
// Use the metadata values as needed
[Link]("JDK Version: " + jdkVersion);
[Link]("OS Name: " + osName);
[Link]("OS Arch: " + osArch);
[Link]("Build Timestamp: " + buildTimesta
mp);
[Link]("Commit Hash: " + commitHash);
} catch (IOException e) {
[Link]();
}
}
}
Example 6.5: Accessing the manifest metadata programmatically

In this code snippet, the JarFile class is used to open the JAR file, and the
Manifest object is obtained from the JAR file. The Attributes object is then
retrieved from the manifest, and the metadata values are accessed using the
appropriate attribute keys.
Your Java application can efficiently capture and access metadata by using
either a Java properties file or the JAR's manifest. These metadata values
provide critical details like the JDK version, OS details, build timestamp,
and SCM commit hash, which can be used for version tracking, debugging,
or giving users context-specific information.

Understanding artifact
An artifact in the context of software development is a file or a collection of
files generated during the software build and deployment process. It
typically consists of the compiled code, libraries, configuration files, and
other resources necessary to distribute or run a software application.
In Java, build tools like Apache Maven or Gradle are commonly used to
create artifacts. These build tools manage Java project compilation,
packaging, and distribution, generating artifacts that are easier to deploy or
share.
To store content in an artifact for a Java project, you typically need to
follow these steps:
1. Define the project structure: Follow the conventions of the build
tool you have chosen and structure your Java project accordingly.
For example, Maven follows a standard directory structure, with
source code files in the src/main/java directory.
2. Configure build tool: Set up the build tool (e.g., Maven or Gradle)
configuration file ([Link] for Maven, or [Link] for Gradle)
to define the project's dependencies, build steps, and packaging
options.
3. Specify dependencies: Declare external dependencies your Java
project needs. Your project may have dependencies on libraries or
other types of artifacts. The download and inclusion of these
dependencies during the build process will be handled by the build
tool.
 4. Build the project: Run the build command provided by your build
tool (e.g., mvn package for Maven or gradle build for Gradle). This
command triggers the compilation of your Java source code, tests,
and packaging of the artifact.
5. Retrieve the artifact: The artifact will be created in a specified
output directory following a successful build. The location and
format of the artifact depend on the build tool configuration. For
example, Maven typically generates a Java Archive (JAR) file in
the target directory.
6. Store and distribute the artifact: The generated artifact should be
kept in a suitable place, like a version control system or a local
repository. You may want to use a repository manager like Nexus or
Artifactory if you want to share the artifact with other developers or
deploy it in production environments.
By following these steps, you can create an artifact for your Java project,
allowing you to package and distribute your application or library
effectively. The generated artifact encapsulates the necessary files and
resources required to run or share your Java project with others.
Let us consider an example of creating an artifact for a simple Java project
using Apache Maven:
1. Project structure: Assume you have a Java project with the
following directory structure:
my-java-project
└── src
└── main
└── java
└── com
└── example
└── [Link]
 The [Link] file contains a simple Java program
that prints "Hello, World!".
2. Configure Maven: Create a [Link] file (Example 6.6) in the root
directory of your project with the following content:
<project xmlns="[Link]
xmlns:xsi="[Link]
xsi:schemaLocation="[Link]
[Link]
<modelVersion>4.0.0</modelVersion>
<groupId>[Link]</groupId>
<artifactId>my-java-project</artifactId>
<version>1.0.0</version>
<dependencies>
<!-- Add any required dependencies here -->
</dependencies>
<build>
<sourceDirectory>src/main/java</sourceDirectory>
<plugins>
<plugin>
<groupId>[Link]</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.11.0</version>
<configuration>
<archive>
<manifest>
 <mainClass>[Link]</main
Class>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
</project>
Example 6.6: Maven POM file ([Link]) with Maven configurations

3. Build the project: Open a command prompt or terminal, navigate to

the root directory of your project, and run the following command:
mvn package
This command triggers the Maven build process. Maven will
compile your Java source code, run tests (if any), and package the
artifact.
4. Retrieve the artifact: After a successful build, Maven will create
the artifact (a JAR file) in the target directory of your project. In
this case, you will find a file named [Link] in
the target directory.
5. Store and distribute the artifact: You can store the generated
artifact in a local repository or distribute it to other developers or
environments. If you plan to share it with others, you might consider
using a repository manager like Nexus or Artifactory.
The resulting artifact, [Link], can be executed by
running java -jar [Link]. It contains the compiled
[Link] file and can be shared and deployed to run your Java
application on compatible systems.
Package management
Package management refers to the process of acquiring, organizing, and
maintaining software packages or libraries. It involves tasks such as
resolving dependencies, fetching packages from repositories, and managing
versioning and updates. To make sure that the required software
components are available for use in projects, package management tools
handle the storage, distribution, and installation of packages. It offers a
more efficient way to handle the acquisition, versioning, and organization
of external software packages.
Projects in the software development industry frequently rely on various
external libraries and modules to carry out particular tasks or provide
additional functionality. These dependencies may have their own
dependencies and may originate from various sources, such as local files or
remote repositories. By automating processes like dependency resolution,
download, and configuration, package management tools make handling
dependencies simpler. These tools make sure that dependencies are
obtained in the correct versions and that any conflicts or inconsistencies are
resolved.
The exact versions of dependencies can be specified by developers to
ensure consistent builds across different environments. This enhances
collaboration and makes it easier to recreate the development environment,
producing results that are more reliable and reproducible.
Additionally, package management simplifies the process of updating
dependencies. Package managers can check for new versions, security
patches, or bug fixes, allowing developers to easily upgrade their projects
and benefit from the latest improvements.
Popular package management tools for Java projects include Maven and
Gradle. These tools provide declarative configuration files where
developers can define their project's dependencies and specify other build
settings.
By leveraging package management, developers can spend less time
manually managing dependencies and more time on the logic and
functionality of their projects. Package management streamlines the
process, enhances productivity, and improves the overall development
experience. In next section, we will discuss about dependency management.

Dependency management
Dependency management in Java projects was introduced with the advent
of build automation tools like Apache Maven in early 2000s. Maven,
released in 2004, popularized the concept of declarative dependency
management for Java projects. Before Maven, developers had to manually
download, configure, and manage dependencies for the Java projects. This
process involved finding the required libraries, downloading them, setting
up the classpath, and resolving any conflicts between different versions of
dependencies.
Maven introduced a standardized way to manage dependencies by utilizing
a declarative approach. Developers could specify the dependencies their
project required in a central configuration file called [Link]. POM file
contained the project’s metadata, including its dependencies, build settings,
and other project specific information.
Since then, other build automation tools like Gradle, which was introduced
in year 2008, have also gained popularity in the Java ecosystem. Gradle
provides enhanced dependency management capabilities while offering
flexibility and extensibility in build configuration. These tools have become
integral parts of modern Java development, enabling developers to manage
dependencies efficiently and streamline the build process.
Dependency management and dependency resolution continue to be a
challenge for many even though they are fundamental features offered by
Maven since their inception and are also fundamental features in Gradle.
Although the rules for declaring dependencies are straightforward, you
might find that published metadata contains invalid, misleading, or missing
constraints.
In this section we will see dependency management in Apache Maven and
Gradle both. We will begin with Maven, as it is the build tool that defines
the artifact metadata using the [Link] file format.

Dependency management with Apache Maven

Dependencies are identified by three required elements: groupId, artifactId,
and version. These elements are collectively known as GAV coordinates
(also known as Maven coordinates). GAV, stands for groupId, artifactId,
and version. Sometimes, you may find dependencies that define a fourth
element named classifier.
One by one, let us understand them. The groupId represents the
organization or group that is responsible for the [Link] typically follows
a reverse domain name convention such as “[Link]” or
“[Link]”. Group IDs help to ensure uniqueness and prevent
naming conflicts between artifacts from different resources. Both artifactId
and version are straightforward; the former defines the “name” of the
artifact, and the latter defines a version number. Many different versions
may be associated with the same artifactId. The classifier adds another
dimension to the artifact, although optional. Classifiers are often used to
differentiate artifacts that are specific to a particular setting such as the
operating system or the Java release.
To define dependencies for your Java project using Apache Maven, you
need to specify them in the [Link] file, which is the configuration file for
your Maven project. Here is how you can define dependencies:
1. Open the [Link] file in the root directory of your Maven project.
2. Locate the <dependencies> element within the <project> element.
If the <dependencies> element does not exist, you can add it within
the <project> element.
 3. Within the <dependencies> element, add <dependency> elements
for each external library or artifact that your project depends on.
Each <dependency> element consists of the following sub-
elements:
• <groupId>: The group or organization that provides the
dependency.
• <artifactId>: The unique identifier for the dependency.
• <version>: The specific version of the dependency to use.
Here is an example (Example 6.7) of adding a dependency for the Apache
Commons Lang library:
<dependencies>
<dependency>
<groupId>[Link]</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.12.0</version>
</dependency>
</dependencies>
Example 6.7: Dependency for Apache Commons Lang library

Add the above dependency for the Apache Commons Lang library on the
[Link] file and save the [Link] file. On the next run, the following
things will happen:
1. Maven will automatically resolve and download the specified
dependencies from remote repositories (such as Maven Central)
when you build your project.
2. After adding the dependencies, you can use the libraries or artifacts
within your Java code by importing the necessary classes and
methods.
 3. When you run the Maven build command (mvn package), Maven
will download the specified dependencies and include them in the
classpath during compilation and execution of your Java project.
It is important to ensure that the specified dependencies are available in the
configured Maven repositories, such as Maven Central. If the dependencies
are not found in the repositories you have configured, you may need to add
additional repository declarations in your [Link] file to access them.
Dependencies listed in this way are known as direct dependencies, as they
are explicitly declared in the POM file. This classification holds true even
for dependencies that may be declared in a POM that is marked as a parent
of the current POM. Parent POM is just like another [Link] file except
that your POM marks it with a parent/child relationship by using the
<parent> section. In this way, configuration defined by the parent POM can
be inherited by the child POM. We can inspect the dependency graph by
running the mvn dependency:tree command, which resolves the
dependency graph and prints it out:
[INFO] --- maven-dependency-plugin:3.11.0:tree (default-cli) @ your-p
roject ---
[INFO] [Link]:your-project:jar:1.0.0
[INFO] +- [Link]:commons-lang3:jar:3.12.0:compile
[INFO] | \- commons-io:commons-io:jar:2.12.0:compile
[INFO] +- [Link]:spring-core:jar:6.0.5:compile
[INFO] | \- [Link]:spring-jcl:jar:6.0.5:compile
[INFO] +- [Link]:[Link]-api:jar:4.0.1:provided
[INFO] \- junit:junit:jar:4.13.2:test
In this example, the output shows the dependency tree for your-project
project. Each line represents a specific dependency, and the indentation
indicates the dependency's level in the tree. Here is a breakdown of the
example output:
 The first line indicates the project itself: [Link]:your-
• project:jar:1.0.0.
• The subsequent lines represent the project's dependencies and their
relationships with the project. For example, the first dependency is
[Link]:commons-lang3:jar:3.12.0:compile, which is a
direct dependency of the project.
• The lines preceded by +- represent direct dependencies, while lines
preceded by | or \ represent transitive dependencies.
• Each dependency line provides information such as group ID,
artifact ID, packaging type (e.g., jar), version, and scope (e.g.,
compile, provided, or test).
By examining the mvn dependency:tree output, you can understand the
complete dependency hierarchy of your project, including the transitive
dependencies brought in by your direct dependencies. This information
helps in troubleshooting dependency conflicts, identifying redundant
dependencies, and ensuring that the correct versions of dependencies are
being used in your project. Keep in mind that the values and versions you
will see might be different.

Dependency management with Gradle

As mentioned earlier, Gradle builds on top of the lessons learned from
Maven and understands the POM format, allowing it to provide dependency
resolution capabilities similar to Maven. Gradle also offers additional
capabilities and finer-grained control. Let’s have a look at what Gradle
offers.
First, you must select the DSL for writing the build file. Your options are
the Apache Groovy DSL or the Kotlin DSL. We’ll continue with the
former, as Groovy has more examples. The next step is picking the format
for recording dependencies, for which there are quite a few; the most
common formats are a single literal with GAV coordinates, such as this:
'[Link]:commons-collections4:4.4'
The Map literal splits each member of the GAV coordinates into its own
element, such as this:
group: '[Link]', name: 'commons-collections4', version:
'4.4'
Gradle chose to go with group instead of groupId, and name instead of
artifactId, though the semantics are the same.
The next order of business is declaring dependencies for a particular scope
(in Maven’s terms), though Gradle calls this configuration, and the behavior
goes beyond what scopes are capable of.
Assuming the java-library plug-in is applied to a Gradle build file, it
provides access to several default configurations that can be used to manage
dependencies and customize the build process for a Java library. These
configurations include:
1. Implementation configuration: The implementation configuration
is used to declare dependencies that are required for compiling and
running the main source code of the Java library. Dependencies
specified in this configuration are not exposed to consumers of the
library. This configuration is suitable for internal dependencies that
are not intended to be used directly by other projects.
2. API configuration: The api configuration is similar to the
implementation configuration but with the added functionality of
exposing the declared dependencies to consumers of the library. It is
useful for dependencies that are part of the library's public API and
need to be accessible to other projects that use the library.
3. testImplementation configuration: The testImplementation
configuration is used to specify dependencies required for compiling
and running the unit tests in the test source code. Dependencies
declared in this configuration are not exposed to consumers of the
library.
4. testRuntimeOnly configuration: The testRuntimeOnly
configuration is used for dependencies that are required at runtime
 only for running the unit tests. These dependencies are not needed
for compiling the tests.
5. runtimeOnly configuration: The runtimeOnly configuration is
used for dependencies that are required only at runtime and not for
compiling or testing the Java library.
These default configurations provided by the java-library plugin give you
convenient options to declare and manage dependencies for your Java
library. You can modify these configurations and add custom configurations
to suit the specific requirements of your project.
In more recent versions of Gradle, the compileOnly and testCompileOnly
configurations have been deprecated and replaced with the implementation
and testImplementation configurations respectively. The compileOnly and
testCompileOnly configurations were previously used to declare
dependencies that were required for compiling but not for runtime.
However, with the shift towards more advanced dependency management,
the implementation and testImplementation configurations now cover
both compile-time and runtime dependencies.
The classpaths follow a hierarchy, just like Maven. Every dependency set in
the api or implementation configurations is also available for execution
because the compile classpath can be consumed by the runtime classpath.
This classpath can be used by the test compile classpath as well, allowing
test code to see production code. The test runtime classpath consumes both
the runtime and test classpaths, allowing test execution access to all
dependencies defined in all of the aforementioned configurations.
As with Maven, dependencies can be resolved from repositories. In contrast
to Maven, where both Maven Local and Maven Central repositories are
always accessible, in Gradle we must explicitly define the repositories from
which dependencies may be consumed. You can define repositories in
Gradle that adhere to the Ivy layout, the standard Maven layout, or even
local directories with a flat layout. It also provides conventional options to
configure the most commonly known repository, Maven Central. We will
use mavenCentral for now as our only repository. Putting together
everything we have seen so far; we can produce a build file with content
like the following (Example 6.8):
plugins {
id 'java-library'
}
repositories {
mavenCentral()
}
dependencies {
api '[Link]:commons-collections4:4.4'
}
Example 6.8: Gradle build file with dependencies

In this build file:

• The java-library plugin is applied, which sets up the project as a
Java library module.
• The repositories block specifies that the dependencies should be
resolved from Maven Central repository.
• The dependencies block declares the dependency on
[Link]:commons-collections4:4.4. The API
configuration is used to expose this dependency to the consumers of
the library.
You can save this content in a file named [Link] in the root directory
of your project. When you build your project using Gradle, it will fetch the
required dependency from Maven Central and make it available for your
project.
When you run the command gradle dependencies --configuration
compileClasspath, Gradle will generate a report that displays the resolved
dependencies for the compileClasspath configuration. The output will list
all the dependencies required for compiling the source code of your project.
Following is the sample output:
compileClasspath - Compile classpath for source set 'main'.
+--- [Link]:commons-collections4:4.4
\--- some-other-library:1.0.0
\--- another-dependency:2.0.0
BUILD SUCCESSFUL in 1s
1 actionable task: 1 executed
In this example, the output shows the resolved dependencies for the
compileClasspath configuration. The [Link]:commons-
collections4:4.4 dependency is listed as a direct dependency, and it does not
have any transitive dependencies in this case.
Now update the build file with the commons-beanutils:commons-
beanutils:1.9.4 dependency added to the implementation configuration.
This means it will be available during both compile-time and runtime.
plugins {
id 'java-library'
}
repositories {
mavenCentral()
}
dependencies {
api '[Link]:commons-collections4:4.4'
implementation 'commons-beanutils:commons-beanutils:1.9.4'
}
Once you save the updated build file, you can run the gradle dependencies -
-configuration compileClasspath command to generate an updated report
of the resolved dependencies for the compileClasspath configuration,
which will include the newly added commons-beanutils dependency.
gradle dependencies --configuration compileClasspath
compileClasspath - Compile classpath for source set 'main'.
+--- [Link]:commons-collections4:4.4
+--- commons-beanutils:commons-beanutils:1.9.4
\--- some-other-library:1.0.0
\--- another-dependency:2.0.0
BUILD SUCCESSFUL in 1s
1 actionable task: 1 executed
In this example, the output shows the resolved dependencies for the
compileClasspath configuration. The [Link]:commons-
collections4:4.4 and commons-beanutils:commons-beanutils:1.9.4
dependencies are listed as direct dependencies, and there is also a transitive
dependency, some-other-library:1.0.0, which itself depends on another-
dependency:2.0.0.
Now update the build file with the transitive = false configuration added to
both the [Link]:commons-collections4:4.4 and commons-
beanutils:commons-beanutils:1.9.4 dependencies. By setting transitive =
false, you explicitly exclude any transitive dependencies that would
otherwise be pulled in by default. This allows you to have finer control over
the dependencies in your project.
plugins {
id 'java-library'
}
repositories {
 mavenCentral()
}
dependencies {
api('[Link]:commons-collections4:4.4') {
transitive = false
}
implementation('commons-beanutils:commons-beanutils:1.9.4') {
transitive = false
}
}
Running the dependencies task once more now shows the resolved
dependencies for the compileClasspath configuration. The
[Link]:commons-collections4:4.4 and commons-
beanutils:commons-beanutils:1.9.4 dependencies are listed as direct
dependencies, without any transitive dependencies included.
gradle dependencies --configuration compileClasspath
compileClasspath - Compile classpath for source set 'main'.
+--- [Link]:commons-collections4:4.4
\--- commons-beanutils:commons-beanutils:1.9.4
BUILD SUCCESSFUL in 1s
1 actionable task: 1 executed
Please note that excluding transitive dependencies should be done carefully,
as it may result in missing dependencies required for the proper functioning
of your project. Make sure to thoroughly test your project after excluding
transitive dependencies to ensure everything works as expected.
Dependency management is a critical aspect of software development, and
Gradle offers a powerful alternative to Maven dependencies efficiently.
With its flexible DSL, incremental build capabilities, extensive plugin
ecosystem, and native language support, Gradle simplifies the process of
managing dependencies in your project.

Dependency management in containers

The way software applications are developed, deployed, and managed has
been revolutionized by containerization. Developers can package their
applications with their dependencies and run them consistently in various
environments by using containers. For stability, portability, and scalability
to be preserved, dependencies must be managed within containerized
applications. In this section, we will explore the basics of dependency
management for containers, their importance, and best practices.

Basics of dependency management in containers

Dependency management in containers refers to handling external libraries,
frameworks, and system-level dependencies needed by an application
within a containerized environment. Containers encapsulate applications
and their dependencies in isolated environments, which ensures that they
run consistently regardless of the underlying infrastructure. Effective
dependency management makes sure that all required dependencies are
available inside the container and are properly versioned to prevent conflict
and compatibility problems.
The benefits of dependency management in containers are as follows:
• Portability: Applications become extremely portable by handling
dependencies within containers. Because containers encapsulate
both the application and its dependencies, they allow the
deployment of applications across various environments without
having to worry about compatibility or external dependencies.
• Isolation: Every container has its own isolated environment with all
necessary dependencies, thanks to dependency management in
 containers. This isolation prevents dependencies from conflicting
and offers a controlled environment for running applications.
• Reproducibility: Dependency management enables the
reproducibility of containerized applications at different stages of
the software development lifecycle. Developers can specify the
exact versions of dependencies and ensure consistent behavior
throughout development, testing, and production environments.
• Scalability: When dependencies are managed properly, containers
enable easier scaling. Managing dependencies ensures that each
container has the required dependencies to operate effectively at
scale because containers can be easily replicated and scaled
horizontally.

Best practices for dependency management in

containers
Some best practices for dependency management are as follows:
• Use container images: Start with a well-defined base container
image containing the required runtime environment and system-
level dependencies. Container images provide running applications
with a consistent and repeatable environment.
• Container orchestration: Leverage container orchestration
platforms like Kubernetes to manage and scale the deployment of
containers. Container orchestration platforms make it easier to
manage dependencies by providing mechanisms for managing them
within containerized applications.
• Dependency locking: Utilize dependency locking mechanisms to
ensure reproducibility. Tools like Docker's [Link] or
package managers such as npm's [Link] allow you to
fix the versions of dependencies, preventing unintended upgrades or
compatibility issues.
• Minimize dependencies: Keep the number of dependencies in your
containerized applications to a minimum. Assess each dependency's
 requirement and consider using lightweight alternatives whenever
possible to reduce the attack surface, optimize performance, and
simplified maintenance.
• Continuous monitoring and updates: Regularly monitor and
update dependencies to ensure security patches, bug fixes, and
performance improvements. Implement automated processes to keep
dependencies up to date while maintaining backward compatibility.
Now, let us consider a scenario where we have a containerized Java
application using Spring Boot. The Dockerfile for this application could
look like this (Example 6.9):
FROM adoptopenjdk:17-jre-hotspot
WORKDIR /app
COPY target/[Link] .
CMD ["java", "-jar", "[Link]"]
Example 6.9: Dockerfile for containerized Java application

In this example, we define a base container image (adoptopenjdk:17-jre-

hotspot) that includes the Java runtime environment. We copy the compiled
JAR file ([Link]) into the container and specify the
command to run the application. The application's dependencies, such as
Spring Boot libraries, are managed through the project's build tool (e.g.,
Maven or Gradle) and resolved during the build process. The container
image contains all the necessary dependencies, ensuring the application
runs smoothly within the isolated container environment.
Dependency management is a critical aspect of containerized application
development. You can ensure the portability, isolation, reproducibility, and
scalability of containerized applications by skillfully managing
dependencies. You can improve the overall reliability and maintainability of
your containerized applications by following best practices like using
container images, dependency locking, minimizing dependencies, and
continuous monitoring and updates.
Containers provide an ideal environment for deploying applications along
with their dependencies, enabling consistent and efficient software delivery.
Whether you are using Docker, Kubernetes, or other containerization
technologies, understanding and implementing effective dependency
management practices will contribute to the success of your containerized
application projects.
Build resilient, scalable, and portable applications by leveraging the power
of containers and robust dependency management techniques. By
effectively managing dependencies, you can unlock the full potential of
containerization, accelerate your software development lifecycle, and
deliver high-quality applications in a streamlined and efficient manner.

Artifact publication
We have talked about how to resolve dependencies and artifacts up to this
point, often from repositories. But what exactly is a repository, and how do
you publish artifacts to it? Artifact publication is a crucial aspect of Java
software development, enabling efficient distribution and sharing of
software components. By publishing artifacts, developers can make their
libraries, frameworks, and applications available to others, fostering
collaboration and reusability. This section will explore the concept of
artifact publication in Java, its importance, and the popular tools and
practices used for artifact publication.

Artifact publication and its benefits

An artifact repository is primarily file storage that keeps track of artifacts.
Each published artifact has metadata collected by a repository, which is then
used to provide extra features like search, archiving, access control lists
(ACLs), and others. Tools can use this metadata to provide additional
features such as vulnerability scanning, metrics, categorization, and more.
Artifact publication involves creating and sharing software artifacts, which
are packaged, versioned, and distributable code units. These artifacts
typically include compiled binaries, configuration files, documentation, and
metadata necessary for consumption by other developers or systems. The
process of artifact publication involves packaging the code, adding relevant
metadata, and making it accessible through a repository.
Artifact publication brings several benefits to Java software development:
• Collaboration and reusability: Publishing artifacts enables
collaboration among developers and encourages the reuse of
existing software components. By sharing artifacts, developers can
leverage each other's work, saving time and effort by avoiding
redundant development.
• Version management: Artifact publication allows for proper
versioning of software components. This ensures that consumers can
choose specific versions of artifacts and facilitates dependency
management, making it easier to maintain compatibility between
different libraries and frameworks.
• Distribution and deployment: Publishing artifacts to a repository
simplifies the process of distributing and deploying software.
Consumers can access artifacts through a repository and include
them in their projects, reducing the complexities of manual
integration and configuration.

Best practices of artifact publication

To ensure effective artifact publication in Java, consider the following best
practices:
• Versioning: Follow a clear versioning scheme for your artifacts to
maintain consistency and manage dependencies effectively. Use
semantic versioning (e.g., [Link]) to indicate
compatibility and changes in your artifacts.
• Documentation: Provide comprehensive documentation along with
your artifacts to help consumers understand their purpose, usage,
and dependencies. Clear and up-to-date documentation enhances the
usability and adoption of your published artifacts.
 • Repository management: Choose a reliable and scalable artifact
repository manager, such as JFrog Artifactory or Nexus Repository
Manager. These tools offer advanced features for artifact
management, access control, and distribution.
• Continuous integration and deployment (CI/CD): Integrate
artifact publication into your CI/CD pipeline to automate the
process. By leveraging tools like Jenkins or GitLab CI/CD, you can
automatically build, test, and publish artifacts with every code
change.
For Maven dependencies, or those that can be resolved using GAV
coordinates, we can use either local or remote repositories. Maven keeps
track of dependencies that have been resolved in a configurable directory
that can be found in the local file system. These dependencies could have
been added directly by the Maven tool or downloaded from a remote
repository. This directory is commonly known as Maven Local and can be
found by default at .m2/ repository. This location is configurable. Remote
repositories, on the other hand, are managed by repository software,
including Sonatype Nexus Repository, JFrog Artifactory, and others. Maven
Central, the canonical repository for resolving artifacts, is the most well-
known remote repository.
Now, we will discuss how to publish artifacts to local and remote
repositories.

Publishing to Maven Local

The Maven Local repository is a local cache on your development machine
that stores artifacts downloaded during the build process. By publishing
your artifacts to Maven Local, you make them available for other projects
on your local machine. Here is how you can publish artifacts to Maven
Local.
When developing Java projects, it is often necessary to publish your
artifacts to a local Maven repository. This allows other projects on your
machine to use the locally built artifacts as dependencies. There are
different methods to publish an artifact to the Maven Local repository.
Following are two major ways to do this:
1. Manual installation: The simplest way to publish an artifact to
Maven Local is by manually installing it using the Maven install
goal. Follow these steps:
Step 1: Configure your project: Ensure that your project has a valid
Maven [Link] file. The [Link] should contain the necessary
project information, such as group ID, artifact ID, version, and
dependencies.
Step 2: Build and install the artifact: Run the following Maven
command in the root directory of your project:
mvn install
This command compiles your project, runs tests (if any), and installs
the generated artifact into your local repository (~/.m2/repository).
Step 3: Verify the publication: Check the ~/.m2/repository
directory to ensure that your artifact has been published successfully
to Maven Local.
Now, other projects on your local machine can reference your
published artifact in their [Link] files using the appropriate group
ID, artifact ID, and version.
2. Maven install plugin: Another approach is to configure the Maven
Install Plugin in your project's [Link] file. Add the following
configuration to the <plugins> section (Example 6.10) :
<build>
<plugins>
<plugin>
<groupId>[Link]</groupId>
<artifactId>maven-install-plugin</artifactId>
<version>3.1.1</version>
 <executions>
<execution>
<id>install-artifact</id>
<phase>install</phase>
<goals>
<goal>install-file</goal>
</goals>
<configuration>
<file>${[Link]}/${[Link]
factId}-${[Link]}.jar</file>
<groupId>[Link]</groupId>
<artifactId>my-artifact</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
Example 6.10: Maven install plugin in [Link]

This configuration installs a specific file to the local repository

during the install phase. Adjust the <file>, <groupId>,
<artifactId>, <version>, and <packaging> values to match your
project's details.
If you are using Gradle as your build tool, you can utilize the
Shadow plugin to create a fat JAR with dependencies and publish it
to the local repository. Add the configuration like below to your
[Link] file:
plugins {
id '[Link]' version '7.0.0'
}
shadowJar {
[Link]('all')
}
artifacts {
archives shadowJar
}
This configuration creates a fat JAR using the Shadow plugin and
adds it as an artifact. Running gradle install will publish the fat JAR
to the local repository.
Alternatively, you can use the Gradle Maven Plugin to publish an
artifact to Maven Local. Add the following configuration to your
[Link] file as shown in Example 6.11 :
plugins {
id 'maven-publish'
}
group = '[Link]'
version = '1.0.0'
publishing {
repositories {
 mavenLocal()
}
publications {
maven(MavenPublication) {
from [Link]
}
}
}
Example 6.11: Gradle Maven Plugin to publish an artifact

This configuration sets up the Maven Publication and publishes the

JAR created by the java component to the Maven Local repository
using the mavenLocal() repository.

Publishing to Maven Central

Maven Central is a widely used repository that hosts a huge collection of
Java artifacts. Publishing your artifact to Maven Central allows developers
worldwide to easily include your library or project as a dependency in their
own projects. You must make sure that you comply with several
requirements before you can publish to Maven Central. These requirements
include:
• Unique group ID: Ensure your project has a unique group ID,
typically in the reverse domain name format (for example,
[Link]).
• Versioning: Follow proper versioning schemes to distinguish
different releases of your artifact (for example, semantic
versioning).
• Code quality: Maintain high code quality, including comprehensive
test coverage and adherence to best practices.
 • Project metadata: Prepare accurate and informative metadata,
including a description, license information, and project website.
Now, let us walk through the steps to publish the artifact to Maven Central:
1. Generate GPG key pair: Maven Central requires artifacts to be
signed using GPG (GNU Privacy Guard) for security purposes.
Generate a GPG key pair by following the instructions provided by
the GPG tool you prefer.
2. Configure Maven settings: Ensure your Maven settings are
correctly configured. Add the following information to your
~/.m2/[Link] file (Example 6.12) :
<settings>
<servers>
<server>
<id>ossrh</id>
<username>YOUR_SONATYPE_USERNAME</username>
<password>YOUR_SONATYPE_PASSWORD</password>
</server>
</servers>
<profiles>
<profile>
<id>ossrh</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<[Link]>gpg</[Link]>
 <[Link]>YOUR_GPG_PASSPHRASE</[Link]
rase>
</properties>
</profile>
</profiles>
</settings>
Example 6.12: ~/.m2/[Link] file

Replace YOUR_SONATYPE_USERNAME,
YOUR_SONATYPE_PASSWORD, and
YOUR_GPG_PASSPHRASE with your respective values.
3. Update POM file: In your project's [Link] file, ensure that you
have the following configurations:
• <distributionManagement>: Add the following distribution
management configuration within the <project> element:
<distributionManagement>
<repository>
<id>ossrh</id>
<url>[Link]
y/maven2/</url>
</repository>
</distributionManagement>
• <scm>: Include Source Control Management (SCM) information
within the <project> element to indicate the project's source
repository.
4. Perform release: To publish your artifact to Maven Central, you
need to perform a release build. This typically involves the
following steps:
 Ensure that your project builds successfully, passes all tests,
a. and is in a stable state.
b. Increment the version number in your [Link] file according
to your versioning strategy.
c. Commit and tag the release in your source control system.
d. Execute the Maven release plugin using the following
command:
mvn clean deploy –Prelease
This command deploys your artifact to Maven Central using
the configured ossrh server in your Maven settings.
5. Close and promote the release: After the deployment completes,
log in to the Sonatype Nexus repository manager and perform the
following steps:
a. Navigate to the Sonatype Nexus repository manager at
[Link]
b. Log in using your Sonatype account credentials.
c. Locate the Staging Repositories section and find the repository
associated with your artifact.
d. Review the contents of the repository to ensure everything is as
expected.
e. Close the repository by selecting it and clicking on the Close
button. This action will initiate a series of checks and
validations.
f. If the checks pass successfully, click on the Release button to
promote the repository contents to Maven Central.
6. Verify publication: Once the release is promoted, your artifact will
be available on Maven Central for others to use. You can verify the
publication by searching for your artifact on the Maven Central
website or by referencing it in another project's dependencies.
Collaboration opportunities and the ability for developers everywhere to
quickly integrate your library or project into their applications are made
possible by publishing your artifact to Maven Central. You can successfully
publish your artifact to Maven Central and make it easily accessible to the
Java community by following the outlined steps, including proper
configuration, signing with GPG, and performing a release. Publishing to
Maven Central helps foster collaboration, promotes open-source
development, and contributes to the Java ecosystem as a whole.
To ensure a smooth publishing process, keep in mind that you must follow
versioning conventions, adhere to best practices, and maintain code quality.
Additionally, to provide users the latest features and improvements, update
your artifact frequently with new releases or bug fixes.

Publishing to Sonatype Nexus repository

It should come as no surprise that the configuration shown in the previous
section also applies here since Sonatype Nexus Repository is used to run
Maven Central; all that needs to be changed to make it work with the Nexus
repository are the publication URLs. There is one restriction, though: a
customized Nexus installation frequently does not fall under the strict
verification rules used by Maven Central. In other words, Nexus has the
option to set the rules for artifact publication. For a Nexus instance running
within your organization, these rules may be relaxed, for example, or they
may be stricter in other areas. In order to publish artifacts to your
organization’s specific Nexus instance, it is a good idea to review the
documentation that is already available at your company.

Publishing to JFrog Artifactory

Another popular option for managing artifacts is JFrog Artifactory. It
provides features that are similar to those of Sonatype Nexus Repository
while also adding other features, like integration with other JFrog Platform
products like Xray and Pipelines. By simply changing the publication URLs
to match the Artifactory instance, the previous publication configuration for
Maven Central will continue to work for Artifactory.
Numerous benefits come with publishing artifacts to JFrog Artifactory, such
as centralized artifact management, version control, dependency resolution,
and simple distribution. It promotes artifact sharing among projects and
organizations and enables effective collaboration among development
teams. In addition, Artifactory provides advanced features like build
integration, access control, artifact promotion, and artifact metadata
management, making it a complete solution for managing your software
artifacts.
By leveraging JFrog Artifactory's robust capabilities, you can streamline
your artifact management process, improve collaboration, and ensure
reliable access to your artifacts throughout the software development
lifecycle.

Conclusion
This chapter has covered a wide range of topics, but the key lesson is that
artifacts alone are not enough to build high-quality software or outperform
the competition. Metadata associated with artifacts typically includes
information about their build time, dependency versions, and environment.
It is possible to use this metadata to determine an artifact's provenance and
make it reproducible. Furthermore, the existence of this metadata can
significantly improve observability, monitoring, and other concerns relating
to the fitness and stability of the build pipeline.
Specific to dependencies, we saw the basics of dependency resolution with
popular Java build tools such as Apache Maven and Gradle. We also
discussed the basics of dependency management of containers.
Finally, we discussed how to publish Java artifacts to the local Maven
repository and the popular Maven Central repository, given that successful
publication necessarily requires adherence to a specific set of guidelines.
Maven Central is the canonical repository but not the only one. Sonatype
Nexus Repository and JFrog Artifactory are well-liked options for
managing artifacts at internal locations, such as your company or
organization.
In the next chapter, we will discuss software security and how we can
protect the binaries.

Join our book’s Discord space

Join the book’s Discord Workspace for Latest updates, Offers, Tech
happenings around the world, New Release and Sessions with the Authors:
[Link]
 CHAPTER 7
Protecting Your Binaries

Introduction
Any comprehensive DevOps rollout must include strong software security
measures. New security breaches are discovered every day, which has
increased awareness of the negative effects of inadequate software security
and led to the development of new security regulations. The entire software
lifecycle, from development through production, is impacted by complying
with these new regulations. As a result, DevSecOps is something that each
software developer and DevOps professional should be familiar with.
You will learn how to assess your product's and organization's risk for
security vulnerabilities in this chapter. Additionally, we will discuss scoring
methods for risk assessment as well as static and dynamic security testing
techniques.
Whatever your role, you will be more equipped to contribute to the security
of the software delivery lifecycle within your organization. But first, let us
dig a little deeper into what happens if you do not focus on security and take
measures to secure your software supply chain.

Structure
In this chapter, we will discuss the following topics:
• Supply chain security compromised
• Software security and why we need it
• DevSecOps
• Shift-left Security approach
• Static and dynamic security analysis
• Comparing SAST and DAST
• Interactive application security testing
• Runtime application self-protection
• Planning for DevSecOps pipeline
• Common vulnerability scoring system
• Quality gate methodology
• Practical applications of quality management
• Implementing security with the quality gate method

Objectives
By the end of this chapter, you will learn how to make software safer. The
chapter starts by examining supply chain security from both vendor and
customer perspectives, emphasizing the importance of safeguarding
software integrity amidst the complexities of the supply chain. From there,
we will dive into the fundamental need for robust software security,
highlighting the potential risks and consequences of overlooking security
measures in today's digital landscape.
Moving forward, we will explore the principles of DevSecOps, a
methodology that integrates security practices into the DevOps workflow.
We will discuss its components, benefits, and challenges in detail; we will
understand the necessity of adopting a comprehensive security approach in
modern software development processes. Within this framework, we will
explore the concept of shifting security left, advocating for the early
integration of security considerations into the development process.
You will also learn how to check for problems in software early through
methods like static and dynamic application security testing. Two other
methods, interactive application security testing and runtime application
self-protection, which help make software more secure, are also discussed.
After that, you will learn how to plan and set up a process that includes
security measures while making software. Tools like SonarQube and
Jenkins will be explored. At the end of the chapter, we will discuss quality
gates and how to do risk management through them.

Supply chain security compromised

In today's interconnected and globalized world, supply chains have become
complex and vulnerable to cyber threats.
Cyberattacks on the supply chain could have a significant impact on both
customers and vendors. Supply chain security breaches can result in data
leaks, financial losses, reputational harm, and even the interruption of vital
services. In order to fully comprehend the effects of such compromises, let
us examine the difficulties with supply chain security from both the vendor
and the customer perspectives.

Supply chain security from vendor perspective

Vendors operate as guardians of the supply chain, orchestrating a symphony
of activities that demand meticulous attention to security. Let us delve into
the challenges vendors often faces and what security measures they can take
to mitigate it:
• Challenge 1: Third-party risks: In today's interconnected business
landscape, vendors often rely on third-party suppliers, partners, and
contractors to fulfil various aspects of their operations. While this
collaboration brings benefits in terms of specialization and
efficiency, it also introduces significant security risks. Any
 compromise in the security measures of a third-party entity can
result in vulnerabilities propagating throughout the supply chain.
○ Security measures: To mitigate third-party risks, vendors must
conduct comprehensive risk assessments of their partners,
implement stringent security requirements in contractual
agreements, and regularly audit their partners' security
practices.
• Challenge 2: Supply chain complexity: Modern supply chains span
multiple geographic locations and involve a myriad of
interconnected systems and technologies. The complexity of these
supply chains makes it challenging to maintain visibility and control
over security measures. Each node in the supply chain may have its
security vulnerabilities, making the entire ecosystem susceptible to
exploitation.
○ Security measures: Implementing a centralized supply chain
security management system can provide vendors with better
visibility and control over security measures. Regular security
audits, robust encryption protocols, and intrusion detection
systems can help strengthen security across the entire supply
chain.
• Challenge 3: Insider threats: Insider threats pose a significant risk to
supply chain security. Malicious actors with privileged access within
a vendor's organization or its partner entities can abuse their
privileges to compromise sensitive data or sabotage operations.
○ Security measures: Vendors should adopt the principle of least
privilege, implement robust access controls, and regularly
monitor user activities to detect and prevent insider threats.

Supply chain security from customer perspective

For customers, supply chain security transcends the mere purchase of a
product. It embodies the trust they invest in the brands they choose. At its
core, supply chain security refers to the comprehensive measures, protocol,
and safeguards that vendors and partners employ to ensure the integrity,
confidentiality and availability of goods and information as they traverse the
complex supply chain web. Let us delve into the challenges customer might
face and what security measures can be taken to mitigate it.
• Challenge 1: Data privacy and compliance: Customers entrust
vendors with their sensitive data, such as personal information and
financial details. They expect vendors to handle their data securely
and comply with data protection regulations. A supply chain security
compromise can lead to data breaches, violating customers' privacy
rights and subjecting the vendor to legal repercussions.
○ Security measures: Customers should conduct thorough due
diligence on vendors' security practices, inquire about their
supply chain security measures, and ensure vendors comply
with relevant data protection regulations.
• Challenge 2: Trust and reputation: Customers place a high value on
trust when selecting vendors for their products and services. A
supply chain security breach can severely impact customer trust and
damage a vendor's reputation. Customers may lose confidence in a
vendor's ability to protect their data and may seek alternatives.
○ Security measures: Vendors should be transparent about their
security measures, promptly communicate security incidents to
customers, and demonstrate a commitment to continuously
improving their security practices.
• Challenge 3: Business continuity and resilience: Customers rely on
vendors to provide uninterrupted services and products. Supply
chain disruptions due to security compromises can result in delayed
deliveries, service outages, and financial losses for customers.
○ Security measures: Vendors should have robust business
continuity and disaster recovery plans in place to ensure rapid
response and recovery in the event of a supply chain security
incident.

Full impact graph of supply chain security

The full impact of supply chain security compromises can be visualized as a
graph, showing the interconnectedness of challenges and consequences:

Figure 7.1: Full Impact graph

Compromises in the security of the supply chain pose significant challenges

for both vendors and customers. In order to protect the digital ecosystem,
vendors must manage third-party risks, improve visibility, and secure
complex supply chains. For customers, trust, data privacy compliance, and
business continuity assurances are crucial factors when selecting vendors.
Vendors can safeguard their reputation and build enduring customer trust by
understanding the full implications of supply chain compromises and
putting strong security measures in place. To create a resilient and secure
digital landscape that can withstand the evolving threat landscape,
collaborative efforts from all supply chain stakeholders are required.

Software security and why we need it

In today's technology-driven world, software plays a pivotal role in
powering various aspects of our lives. From mobile applications and web
services to critical infrastructure, software has become the backbone of
modern society. However, this increased reliance on software also raises a
crucial issue: software security.

Understanding software security

Software security is a multidimensional approach that includes a wide range
of practices and techniques to protect software applications, systems, and
data from potential threats and vulnerabilities it aims to defend against
unauthorized access, data breaches, manipulation, and destruction that could
have detrimental effects on both individuals and organizations.

Need for software security

In today's world, good software security is a big deal for everyone.
Following rules, stopping the theft of ideas, making users feel safe, and
keeping an eye on who's using the software are all key parts of ensuring
software is safe. Let us see how making software secure can help stop
problems and keep things safe in our digital world:
• Keeping sensitive data safe: Numerous software programs deal
with sensitive user data, including financial information, intellectual
property, and personal and financial information. Without strong
security measures, this data is vulnerable to theft or misuse, leading
to severe consequences for individuals and organizations.
• Protection against cyber-attacks: With the increasing frequency
and sophistication of cyber-attacks, software security is essential in
protecting against malicious activities like hacking, malware,
ransomware, and data breaches. For businesses and individuals, a
single security breach can have disastrous consequences by
jeopardizing their trust and financial security.
• Ensuring system reliability: Software vulnerabilities can result in
system failures and crashes, which can have a negative impact on
productivity, disrupt services, and result in substantial financial
 losses. Software developers can improve the stability and reliability
of their applications, minimizing downtime and maintaining a
positive user experience by putting security practices into practice.
• Complying with regulations: Strict regulations and compliance
requirements regarding data protection and privacy apply to many
industries. Legal repercussions, penalties, and reputational harm can
result from not complying with these requirements. Robust software
security ensures compliance and helps maintain a lawful and ethical
business operation.
• Preventing the theft of intellectual property: Theft of intellectual
property is a major concern for businesses and software developers.
Effective security measures protect proprietary algorithms and
software source code from unauthorized access and replication,
preserving the value of original innovations.
• Establishing user trust: If users are confident in the security of the
software, they are more likely to adopt and use it. Building trust with
users and clients through a commitment to software security results
in higher user retention rates and greater customer loyalty.
• Defending against insider threats: Security risks are not always
posed by external parties. Significant risks can also arise from
insider threats, such as dissatisfied staff members or contractors who
have access to sensitive data. By implementing access controls and
monitoring mechanisms, software security helps mitigate these risks.

Key elements of software security

The key elements of software security encompass a comprehensive set of
practices and measures designed to fortify applications against
vulnerabilities and attacks. These elements include:
• Authentication and access management: Keeping certain
functionalities and data accessible to authorized users is one of the
main principles of software security. Passwords, biometrics, and
two-factor authentication, which strengthen access control and
 reduces the risk of unauthorized access, are some of the
authentication methods that help achieve this.
• Encryption: Sensitive information is protected by encryption to
ensure that even if hackers gain access to it, they cannot read or use
it. Data is changed into cipher text through encryption, and the right
encryption key is required to decrypt the cipher text.
• Secure coding practices: The security of software depends heavily
on developers. Software vulnerabilities like those listed in the
OWASP Top 101, which include common issues such as SQL
injection, cross-site scripting (XSS), and insecure deserialization,
can be significantly reduced by adhering to secure coding practices,
such as input validation, avoiding buffer overflows, and using
appropriate error handling.
• Regular patching and updates: Software is not immune to
vulnerabilities, and as new threats appear, developers must quickly
release updates and patches to fix them. Software updates on a
regular basis ensure that security loopholes are fixed and lower the
risk of exploitation.
• Security testing and auditing: A thorough security testing, which
includes penetration testing and vulnerability assessments, can help
identify software flaws before attackers do. Regular security audits
offer valuable information about the infrastructure's and software's
overall security posture. For Java-based projects, integrating security
tools into your build process using a Maven plugin can help
automate the detection of these vulnerabilities. The OWASP
Dependency-Check Maven Plugin is a powerful tool that analyzes
your project dependencies and flags known vulnerabilities in the
libraries you are using. Including this plugin in your Maven build
lifecycle ensures that you are automatically alerted to potential
security risks in third-party components, enabling you to address
them before they become a problem.
• Protocols for secure communication: Secure communication
protocols, such as HTTPS and SSL/TLS, ensures that data is
 transmitted securely between systems so that it cannot be intercepted
by unauthorized parties and remains confidential.

Challenges in software security

Despite significant advancements in software security, challenges persist.
Mentioned here are a few challenges:
• Software complexity: As software becomes increasingly complex,
the number of potential vulnerabilities also rises. Effectively
identifying and eliminating all security risks is difficult due to
complex architectures and large codebases.
• Quick development cycles: Developers may prioritize speed over
security in their quest for quicker updates and releases, leading to
potential oversights. It is crucial to strike a balance between speed
and security so that software remains resilient against evolving
threats.
• Third-party dependencies: Software often relies on third-party
libraries and components. While convenient, these dependencies
could pose security risks if they are not carefully examined and
monitored.
• Older systems: Older software systems might be more vulnerable to
attacks because they lack the robust security features found in
modern applications.
• Human factor: Weak passwords, social engineering attacks, or
improper data handling practices can all result in employees or users
unintentionally causing security breaches.
Software security is an integral part of the digital landscape, safeguarding
our digital infrastructure, individual data, and valuable intellectual property.
As technology continues to evolve quickly, the need for strong software
security has never been more demanding. In an increasingly interconnected
world, businesses and individuals can reduce risks, increase user trust, and
ensure the seamless functioning of their software applications by proactively
implementing security measures. Embracing software security as a
collective responsibility empowers us to create a safer digital ecosystem for
everyone. In next section we will see how we can integrate security in our
software development life cycle.

DevSecOps
Friction is an inevitable consequence when development and security are
separate. Developers may view security as a persistent barrier and
themselves as the agents of change. It is common for security teams to halt
development in order to conduct audits or investigate incidents. At the same
time, Security teams observe developers creating or ignoring the same
issues repeatedly while unable or unwilling to implement clear solutions.
DevSecOps integrates security as a priority within DevOps, placing it at the
center of application development and creating a security-first culture
among everyone in the software pipeline. Security is everyone's
responsibility under the DevSecOps model. This is a first step in lowering
conflict between developers and security engineers.
Similar to DevOps, DevSecOps focuses on removing obstacles between
teams to facilitate open communication. Every product should have security
built in from the beginning, with sacrificing little of speed and agility. Just
as DevOps gave developers more responsibility for testing their own code,
DevSecOps puts the development of secure and compliant code at the top of
every developer's priority.
Every stage of the application development lifecycle is covered by
DevSecOps, which incorporates security best practices while offering
feedback. Security often starts even before the design phase in the form of
training to help developers learn secure coding practices. Security teams
work together with developers. Together with developers, provide training,
document security policies and best practices, and mentor everyone in
developing a security mindset. As DevSecOps practices within an
organization progress, various teams start to see themselves as part of a
single culture where security is the primary objective.
Components of DevSecOps
The components of DevSecOps encompasses a comprehensive set of
practices, tools, and cultural shifts that collectively enhances the security
and agility of software development. Key components includes:
• Automation: A crucial aspect of DevSecOps is automation. In order
to streamline security practices like continuous security testing,
vulnerability scanning, and compliance checks, automated tools and
processes are used. By automating repetitive tasks, teams can
concentrate on more critical security challenges and respond quickly
to new threats.
• Continuous integration and continuous deployment (CI/CD):
CI/CD practices enable frequent and automated code integration,
testing, and deployment. DevSecOps extends these practices by
integrating security checks throughout the CI/CD pipeline. Security
checks are performed at every stage to identify and address
vulnerabilities early in the development process.
• Security testing: A crucial aspect of DevSecOps is security testing.
It includes static application security testing (SAST) and dynamic
application security testing (DAST), which are used to find
security flaws in the application's static code and runtime code,
respectively. These tests help identify and fix vulnerabilities before
they are deployed to production.
• Infrastructure as code (IaC): In the practice of infrastructure as
code, infrastructure is provisioned and managed using code. By
treating infrastructure as code, security configurations can be
codified and version-controlled, lowering the risk of
misconfigurations and fostering consistency across environments.
• Container security: Due to their portability and scalability,
containers are becoming more and more popular for application
deployment. To avoid potential security risks, DevSecOps ensures
that containers are configured securely, scanned for vulnerabilities,
and monitored properly.
 Security auditing and compliance: To make sure that applications
• and systems comply with industry-specific regulations and security
standards, DevSecOps routinely conducts security audits and
compliance checks. This proactive approach lowers the likelihood of
non-compliance and potential legal repercussions.
• Threat modelling: Threat modelling is a technique used to identify
and assess security threats and potential attack vectors early in the
development process. This aids developers in setting priorities for
security initiatives and applying the necessary security controls.
• Security monitoring and incident response: Continuous security
monitoring allows teams to detect and respond to security incidents
promptly. In order to ensure quick and efficient responses to security
breaches, DevSecOps integrates incident response plans into the
development workflow.
• Shift-left culture: DevSecOps encourages teams to address security
concerns as early as possible in the development process by
fostering a shift-left culture. This approach ensures that security
becomes an inherent part of the development mindset and reduces
the likelihood of security issues persisting through production.

Benefits of DevSecOps
The following are a few benefits of DevSecOps:
• Improved software security: DevSecOps ensures that security
vulnerabilities are identified and addressed promptly, reducing the
risk of breaches and data leaks by incorporating security into every
stage of the development process.
• Faster time-to-market: Automated security checks and streamlined
processes enable faster development cycles, accelerating the delivery
of secure and reliable software products.
• Collaborative culture: DevSecOps promotes collaboration and
communication among teams, breaking down silos between
developers, operations, and security professionals. This fosters a
sense of shared responsibility for security.
 • Reduced remediation costs: Early detection and fixes of security
issues reduce the costs later on in the development process of fixing
vulnerabilities.
• Improved compliance: Integrating compliance checks and security
auditing into the development process ensures that applications
adhere to regulatory requirements and security standards.

Challenges of adopting DevSecOps

Even though implementing DevSecOps practices has many benefits, not all
of the steps are simple. Introducing DevSecOps into an organization
presents challenges, from the day-to-day mechanics of securing distributed
applications to the difficult task of cultural change.
Security has typically been cantered around a data centre’s single, well-
defined application perimeter. The enterprise's adoption of DevOps and
cloud native, Microservices-based architecture has led to the
decentralization of applications, which has brought about security
challenges. These applications are made up of Microservices running in
multiple environments, communicating over numerous networks, and
working with data from devices and users all over the world. The attack
surface is thereby both very broad and difficult to define. Counting all
service interactions or all data sent over public and private networks is
nearly impossible.
At the same time, it is simple for security to be neglected when application
ownership changes from teams to larger departments. If they see security as
a problem for a dedicated security team, individual engineering teams won't
invest in it. This tends to move security to the right, into the later phases of
the software development pipeline, where security is more challenging and
less effective.
Shifting left only works when developers are knowledgeable about security.
The developers not only need to be well-versed in secure development
practices but also need enough education to understand the issues they are
tasked to solve. This means training is required, which costs time and
money.
The solution to these issues is to foster a collaborative environment that
encourages quick, continuous iteration with an emphasis on security. That
entails many teams once siloed, such as development, IT operations, and
security, learning to collaborate. Rather than a continual interruption,
security must become a practice incorporated in all aspects of work
throughout the software development pipeline. Instead of causing
interruptions, security teams must offer guidance, turning into continuous
processes like the pipeline's development and operation phases.
Good news! The benefits outweigh the costs. When security is everyone's
responsibility, issues can be solved more quickly and economically,
improving the development process for everyone involved as well as the
customers.

Shift-left Security approach

The significance of software security cannot be overstated as technology
becomes an integral part of our daily lives. The complexity and scope of
cyber threats and attacks are constantly increasing, making it essential to
implement preventative security measures. A new paradigm called Shift-left
Security prioritizes security concerns throughout the entire software
development process.

Understanding the Shift-left Security approach

Embedding security practices and considerations early in the software
development process is the foundation of the Shift-left Security
methodology. In the past, security evaluations were frequently carried out
near the end of the software development process or even software was
deployed. However, this reactive strategy left plenty of room for
vulnerabilities to go undetected, increasing the risks and costs of mitigating
security flaws.
In contrast, the Shift-left Security method, places a strong emphasis on
addressing security concerns as soon as the development process begins,
and the code is written. As a result, there is less chance of important
vulnerabilities slipping through the cracks, and the cost of addressing
security issues is reduced overall. Developers and security teams can work
together to identify and address security issues at an earlier stage.

Key principles of the Shift-left Security approach

The key principles of the Shift-left Security approach revolve around
proactive protection, collaboration, and continuous improvement. The
following are foundational principles:
• Security awareness and education: The Shift-left Security
methodology is based on educating developers and all other parties
involved in the software development process about security best
practices. Developing security-aware teams can help avoid common
coding mistakes and ensure security considerations are ingrained in
the development mindset.
• Integration of security into DevOps: Incorporating security
seamlessly into the DevOps pipeline is the goal of the Shift-left
Security approach. This integration enables continuous security
testing, analysis, and feedback loops, reducing the time between
identifying an issue and implementing a fix.
• Automated security testing: The Shift-left Security methodology is
dependent on automated security testing tools and methods. Two
popular techniques for finding vulnerabilities in the code and
application behavior are static application security testing (SAST)
and dynamic application security testing (DAST).
• Continuous feedback and monitoring: Regularly monitoring
applications and systems in production provides real-time insights
into potential security threats and vulnerabilities. Continuous
feedback allows developers to address emerging security issues
promptly.
Benefits of the Shift-left Security approach
There are various benefits of applying Shift-left Security approach. Some of
the key advantages are:
• Early vulnerability detection: Identifying and addressing security
issues in the early stages of development significantly reduces the
likelihood of severe vulnerabilities in the final product. This saves
time, effort, and costs associated with extensive security fixes later
in the development cycle.
• Shorter time to market: Security assessments are streamlined, and
the entire development cycle is accelerated when security is
integrated throughout the process. As a result, software products can
be developed more quickly and satisfy stakeholder and consumer
demands.
• Improved collaboration: The Shift-left Security methodology
promotes communication between security teams, developers, and
other stakeholders. To create a more reliable and secure application,
teams can share insights, knowledge, and responsibilities by working
together from the beginning.
• Enhanced trust and compliance: The Shift-left Security method
ensures compliance with security and privacy regulations for
companies that operate in regulated industries. Building trust with
customers and clients by showcasing a commitment to software
security improves the organization's reputation.
• Proactive risk mitigation: By proactively addressing security
concerns, organizations can spot potential risks early on and
implement appropriate controls to mitigate them. This helps prevent
security breaches and their associated consequences.
• Cost-effective security techniques: Fixing security vulnerabilities
early in the development process is typically more economical than
addressing them after the product is deployed. Organizations can
more effectively allocate resources thanks to the Shift-left Security
approach.
Challenges and considerations
While the Shift-left Security approach offers numerous benefits, adopting it
requires careful planning and consideration:
• Mindset shifts: A change in organizational culture is necessary to
implement a Shift-left Security strategy. Security must be prioritized
by all parties involved, and everyone must be eager to integrate
security practices into workflows.
• Skill development: To effectively identify and address security
concerns, developers need adequate training in security best
practices and tools. To empower their teams, organizations should
invest in continuous training and skill development.
• Security and development pace must coexist: It is crucial to strike
the right balance between security and development speed. The
development process shouldn't be overly slowed down by security
practices because this could affect time-to-market and
competitiveness.
• Automating integration: While automation is a key component of
the Shift-left Security approach, appropriate security testing tool
selection and integration require thoughtful consideration. Overuse
of automated tools without human validation could result in false
positives or negatives.
The Shift-left Security approach represents a paradigm shift in software
development, advocating for a proactive and security-centric mindset from
the project's inception. Organizations can significantly increase their
software's resilience to cyber threats, reduce vulnerabilities, and lower the
overall cost of security measures by incorporating security considerations
early on. Embracing the Shift-left Security approach is not only a strategic
advantage for businesses but also a responsible step towards protecting user
data, maintaining customer trust, and ensuring a safer digital ecosystem for
all. As the technology landscape continues to evolve, the Shift-left Security
approach will remain a cornerstone of modern software development
practices.
Static and dynamic security analysis
Application security testing is necessary to ensure that your application is
free of vulnerabilities and risks, as well as to reduce the attack surface in
order to prevent cyber-attacks.
Needless to say, applications are widely utilized in almost every industry to
make it easier and easier for consumers to use products and services,
consultations, entertainment, and so on. Furthermore, if you are developing
an application, you must ensure its security from the code phase to
production and deployment. SAST and DAST are two excellent ways to
perform application security testing. Some people choose SAST, some
DAST, and yet others like both conjugations. How can we choose one over
another? For this, we need to compare SAST and DAST to see which is
better in which situation.

Static application security testing

SAST is a testing approach for securing an application by statistically
reviewing its source code to detect all vulnerability sources, including
application vulnerabilities and flaws such as SQL injection.
SAST is also known as white-box security testing, in which the
application's internal components are thoroughly analyzed to identify
vulnerabilities. It is done at the code level in the early phases of application
development before the build is completed. It can also be done once the
application’s components are merged in a testing environment. Furthermore,
SAST is used for an application’s quality assurance.
Additionally, it is carried out with SAST tools while focusing on the code of
an application. These tools scan the app's source code along with all its
components for potential security flaws. They also help in reducing
downtimes and the risk of data compromise.

Understanding how SAST works

SAST tools work something like this:
 • The tool will scan the code while it is at rest to provide a detailed
view of the source code, configurations, environment, dependencies,
data flow, and other elements.
• The SAST tool will check the app's code line by line and instruction
by instruction, comparing it to predefined guidelines. It will examine
your source code for flaws and vulnerabilities such as SQL
injections, buffer overflows, XSS issues, and other issues.
Thus, detecting issues and analysing their consequences will help you in
planning how to fix those issues and improve the application's security.
SAST tools, however, can produce false positives, so you must be well-
versed in coding, security, and design to detect those false positives.
Alternatively, you can modify your code to prevent or reduce false positives.
Error Prone is one of the powerful static analysis tool specifically designed
to catch common mistakes in Java code. Developed by Google, it integrates
seamlessly into the build process, scanning your code for patterns that often
lead to bugs or vulnerabilities. By flagging these issues early, Error Prone
helps developers identify and correct errors before they make it into
production, thereby improving code quality and reducing the likelihood of
security vulnerabilities.

Benefits of SAST
Ensures security during the early stages of development: During the initial
phases of an application's development lifecycle, SAST is essential for
ensuring security. It enables to identify vulnerabilities in your source code
during the coding or designing stage. Additionally, it is simpler to solve
issues when you can identify them early. Nevertheless, if you don't run tests
early to identify issues and let them persist until the end of development, the
build may contain numerous inherent bugs and errors. As a result,
understanding and treating them will become difficult and time-consuming,
further delaying your production and deployment schedule.
By performing SAST, you can avoid spending time and money fixing the
vulnerabilities. Additionally, it can test vulnerabilities on the client and
server sides. All these help in protecting your application and enable you to
build a safe environment for the application and deploy it quickly. Here are
some additional benefits of SAST:
• Quicker and more precise: SAST tools scan your application and
its source code more thoroughly faster than manual code review. The
tools can scan millions of lines of code quickly and precisely to
detect underlying issues. Furthermore, SAST tools continuously
monitor your code for security to preserve its integrity and
functionality while assisting you in quickly mitigating issues.
• Secure coding: When developing code for websites, mobile devices,
embedded systems, or computers, you must ensure secure coding.
When you write robust, safe code from the beginning, you reduce
the chances of your application being compromised. The reason for
this is that attackers can easily target poorly coded applications and
perform harmful activities such as information theft, password theft,
account takeovers, and more. It has a negative impact on your
organization's reputation and customer trust.
Using SAST will help you ensure safe coding practices from the
start and provide a solid foundation for it to thrive throughout its
lifecycle. It will also help you ensure compliance. Furthermore,
Scrum masters can use SAST tools to ensure that safer coding
standards are implemented in their teams.
• Detection of high-risk vulnerabilities: SAST tools can detect high-
risk application vulnerabilities such as SQL injection, which can
impact an application throughout its lifecycle, and buffer overflows,
which can disable it. Furthermore, they identify cross-site scripting
(XSS) and vulnerabilities efficiently. Indeed, strong SAST tools can
detect all of the flaws listed in OWASP's top security threats.
• Easy to integrate: SAST tools are simple to integrate into an
existing application development lifecycle process. They can work
efficiently in development environments, source repositories, bug
 trackers, and other security testing tools with ease. They also provide
a user-friendly interface for consistent testing without requiring
users to go through a high learning curve.
• Automated audits: Manual code audits for security concerns can be
time-consuming. It requires the auditor to understand the
vulnerabilities before they can jump on to examine the code
thoroughly. SAST tools, on the other hand, provide great
performance in terms of examining code frequently with accuracy
and in less time. The tools can also help to improve code security
and speed up code audits.

Disadvantages of the SAST approach

Because you start with the source code, SAST appears to have the most
complete security scanning approach. However, it has fundamental
problems in practice:
• One disadvantage of using a SAST approach is that it can sometimes
lead to a slowdown in the programming process. When developers
are constantly focused on addressing security checks and fixing bugs
identified by SAST tools, it can divert their attention from the core
tasks of writing and refining the application’s functionality. This
heightened focus on security can result in domain-specific issues,
where the developers may neglect or rush through the business logic
and features that are crucial for the application’s success. In other
words, the balance between security and functionality can be
disrupted, potentially impacting the overall quality and relevance of
the software in its intended domain.
• The tools can be problematic. This is especially true if the scanners
have not been fully integrated into your technology stack.
Nowadays, most systems are polyglot. A tool that supports all direct
or indirect technologies is required to obtain a complete list of
known vulnerabilities.
• SAST frequently fully replaces future security tests. However, any
problems that are directly related to an application in operation
 remain undetected.
• Focusing on your source code is insufficient. If possible, the static
scan should analyze the binaries as well as the source code.

Dynamic application security testing

DAST is another testing method that employs a black-box approach,
presuming that the testers do not have access to or knowledge of the
application's source code or core functioning. They test the application from
the outside utilizing the available outputs and inputs. The test resembles a
hacker attempting to obtain access to the application.
DAST aims to observe the application’s behaviour to attack vectors and
identify vulnerabilities remaining in the application. It is done on a
functioning application and needs you to run the application and interact
with it to implement some techniques and perform assessments.
It allows you to discover all security vulnerabilities in your application at
runtime after it has been deployed. You can avoid a data breach by reducing
the attack surface via which real hackers can launch a cyberattack.
Furthermore, DAST can be performed manually or with the help of DAST
tools to implement a hacking approach such as cross-site scripting, SQL
injection, malware, and others. DAST tools can check authentication
difficulties, server settings, logic misconfigurations, third-party risks,
encryption vulnerabilities, and other issues.

Understanding how DAST work

Usually, DAST tools are effortless to use, but they do a lot of complex stuff
behind the scenes to make the testing easy.
• DAST tools are designed to capture as much information about the
application as possible. They crawl each page and extract inputs to
broaden the attack surface.
• Next, they start actively scanning the application. A DAST tool will
send several attack vectors to previously discovered endpoints in
 order to test for vulnerabilities such as XSS, SSRF, CSRF, SQL
injections, and so on. Furthermore, many DAST tools allow you to
create custom attack scenarios in order to detect additional
vulnerabilities.
• Once this step is complete, the tool will display the results. If it
identifies a vulnerability, it quickly provides detailed information
about the vulnerability, including its type, URL, severity, and attack
vector, as well as guidance in resolving the issues.
DAST tools are effective at detecting authentication and configuration
issues that occur during application login. They simulate attacks by
providing certain specified inputs to the application under test. To detect
flaws, the tool compares the output against the expected result. DAST is
widely used in web application security testing.

Benefits of using DAST

DAST focuses on an application's runtime features, providing numerous
advantages to the software development team, including:
• Increased testing scope: Modern applications are complex, with
numerous external libraries, legacy systems, template code, and so
on. Security risks are evolving, and you require a solution that can
provide broader testing coverage, which may not be sufficient if you
only use SAST. DAST can help by scanning and testing all types of
apps and websites, independent of technology, source code
availability, or origins. As a result, adopting DAST can address a
variety of security concerns while also determining how your
application appears to attackers and end users. It will help you in
implementing a comprehensive plan to fix the issues and create a
high-quality application.
• High security in all environments: Since DAST is applied on the
application from the outside, rather than on its underlying code, you
can achieve the highest level of application security and integrity.
Even if you make changes to the application environment, it will
remain secure and fully functional.
 • Tests deployments: DAST tools are used not only to test
applications for vulnerabilities in a staging environment, but also in
development and production environments. This allows you to see
how secure your application is after it has been released. You can
use the tools to scan the application on a regular basis to detect any
underlying issues caused by configuration changes. It may also find
new vulnerabilities that could endanger your application.
• Easy to integrate into DevOps workflows: Many people believe
that DAST cannot be used during the development stage. It was, but
it is no longer valid. Many tools, such as Invicti, may be simply
integrated into your DevOps workflows. So, if you configure the
integration correctly, you may enable the tool to automatically scan
for vulnerabilities and identify security issues during the early
phases of application development. This will improve application
security, eliminate delays in identifying and resolving vulnerabilities,
and save associated costs.
• Aids in penetration testing: Dynamic application security is similar
to penetration testing in that an application is tested for security
vulnerabilities by injecting malicious code or conducting a
cyberattack to see how the program responds.
• With its extensive features, using a DAST tool in your penetration
testing efforts can simplify your work. The tools can streamline the
overall penetration testing by automating the process of finding
vulnerabilities and reporting issues so that they can be fixed
promptly.
• A more comprehensive security overview: DAST has an
advantage over point solutions because it can carefully examine the
security posture of your application. Regardless of their
programming languages, origins, source codes, or other
characteristics, it can test all kinds of websites, applications, and
other web assets.
You can therefore thoroughly comprehend the security state of any software
or application you develop. As a result of greater visibility across
environments, you can even identify risky outdated technologies.

Disadvantages of DAST
Using DAST tools has several disadvantages:
• The scanners are programmed to perform specific attacks on
functional web apps and are typically only adaptable by security
experts with product knowledge. As a result, they leave little room
for individual scaling.
• DAST tools are slow; it can take them many days to complete an
analysis.
• Some security flaws that DAST tools discover relatively late in the
development cycle could have been found earlier via SAST. As a
result, the expenses associated with fixing the related issues are more
than they should be.
• The basis for DAST scanning is known bugs. It takes relatively long
time to scan for new types of attack. Therefore, it is often not
possible to modify the existing tool. If it is doable, it will require in-
depth understanding of the attack vector itself and how to implement
it in the DAST tool.

Comparing SAST and DAST

Both SAST and DAST are types of application security testing. They
inspect apps for vulnerabilities and flaws, assisting in the prevention of
security risks and cyberattacks.
Both SAST and DAST serve the same purpose: to find and highlight
security concerns and help you in fixing them before an attack occurs.
Now, in this SAST vs DAST battle, let us look at some of the primary
differences (Table 7.1) between these two security testing methods:

SAST DAST
Its white-box application security Its black-box application security
testing. testing.
Testing is done from the inside Testing is done from the outside
out (of the applications). in.

It is based on the testing strategy It is based on the approach of

used by developers. The tester is hackers. The tester has no
familiar with the application's knowledge of the application's
design, implementation, and design, implementation, or
framework. frameworks.

It is implemented using static It is implemented on a running

code and does not require any application. It is termed
deployed applications. It is "dynamic" because it scans the
termed "static" because it scans application's dynamic code while
the application's static code for it's running for vulnerabilities.
vulnerabilities.

SAST is performed at the early DAST is performed on a running

stages of application application near the end of its
development. development lifecycle.

It is capable of detecting client- It is capable of detecting

side and server-side vulnerabilities related to
vulnerabilities with accuracy. environment and runtime.
SAST tools work with a wide However, it can only analyze
range of embedded systems and responses and requests in an
code. It cannot, however, application.
discover issues related to
environments and runtime.

It needs source code for testing. It does not require source code
for testing.

SAST is directly integrated into DAST is integrated to a CI/CD

CI/CD pipelines to help pipeline once the app is deployed
developers in constantly and running on a test server or the
monitoring application code. It developer's computer.
covers every stage of the CI
process, including app code
 security analysis via automated
code scanning and testing the
build.

SAST tools comprehensively DAST tools may not give the

scan code to identify precise location of vulnerabilities
vulnerabilities and their precise because they work during
locations, making fix easier. runtime.

Since issues are detected in the Since it is implemented near the

early phases, fixing them is end of the development lifecycle,
simple and less expensive. issues are not detectable until
then. Furthermore, it may not
provide precise locations. All of
this raises the cost of fixing
issues. Simultaneously, it delays
the overall development timeline,
raising overall production costs.

Table 7.1: Comparing SAST and DAST

When to use SAST and DAST

In previous section, we have seen the comparison between SAST and
DAST. Now let’s discuss the use cases where we will be using SAST and
the scenarios where DAST will be used:

When to use SAST

Assume you have a development team that writes code in a Monolithic
environment. When your developers comes up with an update, they
immediately incorporate changes to the source code. Following that, you
compile the application and promote it to the production stage on a
scheduled time.
Vulnerabilities will not appear frequently here, and if they do, it will be after
a long period of time, at which point you can evaluate and patch them. In
this scenario, you should consider using SAST.
When to use DAST
Assume your SLDC has an effective DevOps environment with automation.
Containers and cloud platforms such as AWS can be used. As a result, your
developers can swiftly code updates and use DevOps tools to automatically
compile the code and build containers.
This way, you can use continuous CI/CD to speed up deployment. However,
this may increase the attack surface. Using a DAST tool to scan the entire
application and detect issues could be a brilliant solution for this.

Can SAST and DAST work together

The answer is yes!!!
In fact, using them together will help you understand security issues in your
application from the inside out. It will also enable a DevOps or DevSecOps
process focused on effective and actionable security testing, analysis, and
reporting. Furthermore, this will help reduce vulnerabilities and the attack
surface, as well as mitigate cyberattack issues. As a result, you'll be able to
build a highly secure and strong SDLC.

Interactive application security testing

Interactive application security testing (IAST) evaluates application
performance and identifies vulnerabilities using software tools. IAST uses
an "agent-like" approach in which agents and sensors continually analyze
application functionality throughout automated tests, manual tests, or a
combination of both tests.
The process and feedback occur in real-time in the IDE, CI, or QA
environment, or during production. The sensors have access to the
following:
• Entire code
• System configuration data
• Web components
 • Dataflow and control flow
• Back-end connection data
IAST differs from SAST and DAST in that it operates inside the
application. When compared to source code or HTTP scanning, access to
such a wide range of data expands IAST coverage and provides for more
accurate output.
Some benefits of IAST are:
• Potential issues are identified earlier, allowing IAST to minimize
costs and delays. This is due to the use of a Shift-left approach,
which means it is done early in the project's lifecycle.
• IAST analysis, like SAST, provides detailed data-containing lines of
code, allowing security teams to focus on a specific issue right away.
• With the range of information, the tool has access to, it can precisely
identify the source of flaws.
• Unlike other types of software dynamic testing, IAST can be easily
integrated into CI/CD (continuous integration and deployment)
pipelines. Integrating DAST into CI/CD pipelines is complex and
time-consuming because it tests a fully deployed and running
application. In contrast, IAST integrates more seamlessly into CI/CD
processes as it tests the application internally during its runtime,
allowing for continuous security checks without disrupting the
development flow.
On the other hand:
• IAST tools can cause the application to run slowly. The agents
effectively act as additional instrumentation, causing the code to
perform poorly.
• Because it is a new technology, some flaws may not have been
discovered yet.

Runtime application self-protection

Runtime application self-protection (RASP) is a method of securing an
application from within. The check is done at runtime and generally
involves scanning for suspicious commands as they are executed.
You can examine the full application context on the production machine in
real time using the RASP technique. All processed commands are analyzed
for possible attack patterns. As a result, the purpose of this technique is to
detect both known and unknown security flaws and attack patterns. It delves
into the usage of AI and machine learning (ML) techniques.
RASP tools typically have two operational modes. The first mode of
operation (monitoring) is limited to observing and reporting potential
attacks. The second operating mode (protection) entails putting defensive
measures in place in real time and directly on the production environment.
RASP aims to bridge the gap between application security testing and
network perimeter controls. SAST and DAST do not provide enough
visibility into real-time data and event flows to prevent vulnerabilities from
passing through the verification process or to block threats that were missed
during development.
The main difference is that IAST focuses on finding application
vulnerabilities, whereas RASP focuses on protecting against cybersecurity
threats that exploit these vulnerabilities or other attack vectors.
RASP technology possesses the following advantages:
• RASP complements SAST and DAST by providing an extra layer of
protection once the application has been set up (usually in
production).
• It is easily applied with faster development cycles.
• Unexpected inputs will be inspected and regulated.
• It enables you to respond swiftly to an attack by providing detailed
analysis and vulnerability locations.
However, RASP tools come with certain drawbacks:
 • RASP tools may have a negative impact on application performance
because they are hosted on the application server.
• The new technology may be incompatible with regulations or
internal policies that prohibit the installation of other software or
restricted services.
• You might get the feeling that your application is safer than it is.
Even if the tool has found an issue, it is still means to take your
application offline while it is being fixed.
• RASP is not a replacement for application security testing because it
cannot provide comprehensive protection.
While RASP and IAST have used similar methodologies, RASP does not
perform complete scans and instead runs as a component of the application,
analysing its traffic and activities. They both report on attacks as they occur,
although IAST does so during testing and RASP does so in production.
Note: If you are just getting started with DevSecOps or IT security in
general, the SAST method makes the most sense. This is where the
most serious potential threat can be eradicated with the least amount
of effort. It is also a process that may be applied at all stages of
production line. Only when all components of the system are secured
against known security flaws do the following methods exhibit their
full potential. Following the introduction of SAST, we would adopt
the IAST technique, followed by the RASP approach. This also
ensures that the different teams can expand with the task and that
there are no production impediments or delays.

Planning for DevSecOps pipeline

Adopting a DevSecOps approach is essential for creating secure and
resilient software applications in the modern, fast-paced, and threat-prone
digital landscape. A well-structured DevSecOps pipeline ensures that
security is not an afterthought but an integral part of the software
development lifecycle. Below are the key steps involved in planning for a
successful DevSecOps pipeline, enabling organizations to create a culture of
security, collaboration, and continuous improvement.
1. Define objectives and goals: Clear objectives and goals must be
defined before moving on to the technical aspects of building a
DevSecOps pipeline. Understand the specific security requirements
of your software applications and identify the primary security
concerns. Define key performance indicators (KPIs) to measure
the success of your DevSecOps initiative, such as a decrease in the
number of vulnerabilities shorter response time to security incidents,
or better compliance with industry standards.
2. Assess current security practices: Examine current security
practices: Assess the maturity of your current DevOps processes and
your organization's current security processes. Identify gaps and
areas that require improvement to integrate security seamlessly. To
understand the current security posture of your software and identify
potential threats and vulnerabilities, conduct security assessments.
3. Form a cross-functional team: Form a cross-functional team with
developers, operations personnel, security experts, and other key
stakeholders. This team will collaborate throughout the planning and
implementation of the DevSecOps pipeline, ensuring a holistic
approach that considers the diverse perspectives and expertise of
each member.
4. Establish a security culture: A crucial aspect of a successful
DevSecOps implementation is establishing a security-first culture.
Develop a culture that emphasizes security awareness and fosters
knowledge sharing and prioritizes security considerations at every
stage of the development process. Training sessions and workshops
can be conducted to educate the team about security best practices.
5. Choose the correct technologies and tools: To support the
DevSecOps pipeline, choose and deploy appropriate tools and
technologies. These may include static code analysis tools for early
vulnerability detection, DAST tools for runtime analysis, container
security tools, and infrastructure as code (IaC) tools for secure
 deployments. Pick tools that integrate flawlessly with the existing
development and deployment processes.
6. Integrate security in CI/CD pipeline: Integrate security checks and
testing into your CI/CD pipeline. This makes sure that security
checks are carried out automatically at each stage of the software
development lifecycle. Automated security tests, such as SAST and
DAST, should be conducted during code commits, build processes,
and pre-deployment stages.
7. Implement code review and peer review: Regular code reviews
and peer reviews are essential to identifying potential security issues
and ensuring adherence to security best practices. Encourage
developers to conduct thorough security reviews of each other's
code, promoting accountability and shared responsibility for
security.
8. Implement security testing and vulnerability management:
Continuous vulnerability management and security testing are
necessary for DevSecOps. Scan infrastructure and applications
frequently for vulnerabilities and prioritize fixing them based on
their severity. Implement a process to handle security incidents, and
make sure to respond quickly and address any issues.
9. Monitor and measure performance: Establish a robust monitoring
system to track the performance of your DevSecOps pipeline and
security measures. To enhance the pipeline's effectiveness and
efficiency, continually analyze the data collected, review KPIs, and
make data-driven decisions.
10. Foster continuous improvement: DevSecOps is an iterative
process that necessitates continuous improvement. Conduct post-
mortems and retrospectives on a regular basis to identify problem
areas and make necessary changes. To identify pain points and
address them quickly, encourage feedback from all stakeholders.
It is important to carefully consider organizational goals, current security
practices, and the establishment of a security-first culture when planning a
successful DevSecOps pipeline. By integrating security into the entire
software development lifecycle and fostering collaboration among cross-
functional teams, organizations can build secure and resilient applications
that withstand evolving cyber threats. Continuous monitoring, measurement,
and improvement are critical to maintaining the effectiveness of the
DevSecOps pipeline and ensuring a proactive and robust security posture in
an ever-changing digital landscape. Embracing DevSecOps as a philosophy
and practice enables organizations to build a digital ecosystem that is safer
for their users and strengthens their ability to address security challenges
effectively.

Vulnerability scoring system

A vulnerability scoring system is a standardized methodology that assigns
scores or ratings to security vulnerabilities to help prioritize their
remediation. The scores are used to categorize vulnerabilities based on their
impact and severity, allowing security professionals and developers to
allocate resources effectively and prioritize the most critical challenges.
Understanding a vulnerability scoring system, like the Common
Vulnerability Scoring System (CVSS), is essential for prioritizing security
measures effectively. CVSS provides a standardized way to assess the
severity of security vulnerabilities, considering factors such as the potential
impact on confidentiality, integrity, and availability of systems. By
comprehending the components and methodology of CVSS, organizations
can better manage and mitigate security risks within their software
environments.

Common vulnerability scoring system

The CVSS is one of the most widely used and accepted vulnerability
scoring systems. It was developed by the National Infrastructure
Advisory Council (NIAC) to provide a standardized and impartial way to
assess the severity of security vulnerabilities. The purpose of CVSS is to
categorize security vulnerabilities according to their severity in a general
way. The identified weaknesses are assessed from various points of view.
These elements are weighed against each other to obtain a standardized
number from 0 to 10, with 10 being the most severe.
We can evaluate various weak points in an abstract manner and derive
follow-up actions from them using a rating system like CVSS. The goal is to
standardize how these weak points are handled. You can, therefore, define
actions based on the value ranges.
In principle, CVSS can be described so that the probability and the
maximum possible damage are related using predefined factors. The basic
formula for this is,
Risk = Probability of occurrence × damage

Components of CVSS
CVSS comprises three metric groups:
• Base metrics: These assess the vulnerability's intrinsic qualities and
potential impact. The base metrics include:
○ Attack vector: How an attacker can access the vulnerable
component (e.g., local, adjacent network, or remote).
○ Attack complexity: The level of complexity required to exploit
the vulnerability.
○ Privileges required: The privileges an attacker must possess to
exploit the vulnerability.
○ User interaction: Whether user interaction is necessary to
exploit the vulnerability.
○ Impact on Confidentiality, Integrity, and Availability: The
potential impact on the system's confidentiality, integrity, and
availability.
• Temporal metrics: The time-dependent components of the
vulnerability assessment are brought together in the temporal metrics
group. These temporal metrics are influenced by the elements that
change over time. For example, the availability of tools to support
 vulnerability exploitation may change. These can be exploit code or
detailed instructions. A distinction must be made between a
theoretical vulnerability and one that has been officially confirmed
by a manufacturer. All of these events have an effect on the base
value. Temporal metrics are distinct in that the base value can only
be decreased, not increased. The first rating is meant to represent the
worst-case scenario. When you consider that interests are competing
during the initial assessment of a vulnerability, this has both
advantages and disadvantages. The external framework conditions
have an impact on the initial evaluation. These occur over an
unspecified time period and are unrelated to the actual basic
assessment. Even if an exploit is already in use during the base
values survey, it will not be considered in the primary assessment.
However, the temporal metrics can only be used to reduce the base
value.
This is where the conflict begins. The individual or group who
discovers a security flaw attempts to set the base value as high as
possible. A severe loophole will sell for a higher price and garner
more media attention. As a result, the reputation of the person or
group who discovered this gap grows. The affected company or
project is looking for the exact opposite assessment. As a result, it is
dependent on who discovers the security gap, how the review
process should proceed, and which body conducts the initial
evaluation. The environmental metrics further adjust this value.
• Environmental metrics: For environmental metrics, your own
system landscape is used to assess the risk of a security gap. The
evaluation is adjusted based on the actual situation. In contrast to
temporal metrics, environmental metrics can correct the base value
in both directions. As a result, the environment can lead to a higher
classification and must also be constantly adapted to your own
environment changes.
Consider a security flaw for which a patch is available from the
manufacturer. The mere existence of this modification reduces the
total value in the temporal metrics. However, as long as the patch is
 not activated in your own systems, the overall value must be
drastically corrected upward via environmental metrics. This is
because as soon as a patch is made available, it can be used to
understand the security gap and its consequences better. The attacker
possesses more detailed information that can be exploited, lowering
the resistance of the unhardened systems.
The final score is calculated from the three previously mentioned values at
the end of an evaluation. The final value is assigned to a value group.
However, there is one more point that is frequently overlooked. Many times,
the final score is simply carried over without any individual adjustments
based on the environmental score. This behavior results in a risky evaluation
that is incorrect for the overall system.

CVSS in practice
With CVSS, we now have a system for evaluating and rating software
security gaps thanks to CVSS. CVSS has become the de facto standard
because there are no alternatives; the system has been in use for over ten
years and is constantly being developed.
First and foremost, the basic score represents a purely technical worst-case
scenario. The second component is the evaluation of time-dependent
corrections based on external influences, such as additional discoveries,
tools, or patches for this security gap, which can be used to reduce the value.
In terms of this vulnerability, the third component of the assessment is your
own system environment. With this in mind, the security gap is adjusted in
relation to the actual situation on the job site. Finally, an overall evaluation
is made using these three values, yielding a number ranging from 0.0 to
10.0.
This final value can be used to direct your organization's response to the
security flaws. Everything appears to be fairly abstract at first glance, thus it
takes some effort to obtain a feel for the application of CVSS, which may be
built through experience with your own systems.
Quality gate methodology
In the software development lifecycle, quality gates are predefined
checkpoints or decision points. These checkpoints are designed to evaluate
the quality and readiness of deliverables before they move on to the next
stage of development or deployment. Quality gates are quality assurance
techniques that ensure software meets particular standards at various phases
of development.
Once the quality gate is reached, a project can only be proceeded if all
criteria, or a sufficiently high number of criteria, are met. This ensures that
all project outputs at the time of the quality gate are good enough to
continue working with. The outcomes, on the one hand, and the qualitative
requirements for the results, on the other, can be determined using quality
gate criteria. They can then be used to define the interfaces between specific
project phases. Certain structures, actions, responsibilities, documents, and
resources are required to establish quality gates, and these are summarized
in a quality gate reference process.

Importance of quality gates

Quality Gates offers a range of benefits that contribute to the overall success
of software development efforts. Let us explore why quality gates are so
important:
• Early detection of defects: Quality gates help detect defects and
issues early in the development process, reducing the cost and effort
required to address them.
• Risk mitigation: By assessing and validating the quality of
deliverables before proceeding, quality gates mitigate the risks
associated with deploying faulty software.
• Continuous improvement: Quality gates encourage a culture of
continuous improvement by providing feedback and insights for the
development team to enhance their processes and practices.
 • Predictable development cycle: Implementing quality gates ensures
a more predictable development cycle, with well-defined milestones
for progress.
• Compliance and standards: Quality gates ensure that software
adheres to organizational standards and industry best practices,
promoting compliance with relevant regulations.

Quality gate strategies

There are two primary ways for using quality gates. When designing a
quality gate reference process, a company might use any of two
methodologies indicated below, depending on the objective.
• Quality gates as uniform quality guideline: In the first strategy,
each project must pass through the same quality gates and be
evaluated using the same standards. A minimal amount (if any) of
adaptation of a quality gate reference process that follows this
strategy is allowed. Every project has a quality standard established
in order to achieve at least the same level of quality across all of
them.
Therefore, quality gates can be used as a uniform measure of
progress. By examining which tasks have already passed a particular
quality gate and which have not, we can compare the progress of
different projects. When one project (qualitatively) lags behind
another, management can easily identify this and take appropriate
action. Thus, quality gates are a simple tool that can be used for
managing multiple projects.
• Quality gates as a flexible quality strategy: The second strategy
allows for the selection, arrangement, and number of quality gates or
criteria to be customized to a project's requirements. Thus, quality
gates and standards can be more closely matched to the qualitative
requirements of a project, enhancing the quality of results. However,
this makes comparing different projects more challenging.
Thankfully, similar projects will have comparable quality gates and
can be evaluated using similar criteria.
Implementing quality gates
The process of implementing quality gates varies based on the specific
development methodology and organizational needs. However, the
following general steps can guide the integration of quality gates into the
software development lifecycle:
• Defining quality criteria: Identify the criteria that define the quality
of deliverables at each phase. These criteria may include code
quality, test coverage, security checks, performance benchmarks, and
compliance with coding standards.
• Designing gate triggers: Determine the triggers that will initiate the
evaluation of the deliverables. Triggers may include code commits,
test execution, completion of specific milestones, or before release to
production.
• Creating evaluation metrics: Develop metrics and measurements to
assess the deliverables against the defined quality criteria. Metrics
could be quantitative, such as code complexity or test coverage
percentage, or qualitative, such as adherence to coding standards.
• Establishing thresholds: Set specific thresholds or acceptance
criteria that the deliverables must meet to pass the quality gate. If the
deliverables do not meet these thresholds, they are returned to the
development team for refinement.
• Feedback and continuous improvement: Provide feedback to the
development team based on the results of the quality gate evaluation.
Encourage continuous improvement by addressing identified issues
and incorporating lessons learned into future development efforts.

Practical applications of quality management

In previous section, we have learned about quality gate, its benefits and its
implementation process. Now we will explore the practical applications of
quality management encompass tools and the implementation ensuring the
delivery of high-quality products or services.
Quality management with SonarQube
SonarQube is a widely used open-source platform that offers powerful code
quality analysis and continuous inspection of codebases. It provides a
comprehensive set of static code analysis rules, code smell detection, and
test coverage reports to evaluate the overall quality of the code. In this
example, we will see how to use SonarQube to implement quality gates and
ensure code excellence.
1. Set up SonarQube server
First, we need to set up a SonarQube server. You can download the
SonarQube server from the official website and follow the
installation instructions for your platform.
2. Configure SonarQube quality gates
Once the SonarQube server is running, log in to the SonarQube web
interface and navigate to the Quality Gates section. Here, you can
define quality gate conditions based on various criteria, such as:
Coverage: The minimum code coverage percentage required for the
codebase.
Bugs: The maximum allowed number of bugs in the code.
Vulnerabilities: The maximum allowed number of security vulnerabi
lities in the code.
Code Smells: The maximum allowed number of code smells in the c
ode.
Duplication: The maximum allowed percentage of duplicated code.
For example, let us create a quality gate called "Code Excellence"
with the following conditions:
Code Coverage: At least 80% coverage
Bugs: No more than 10 bugs
Vulnerabilities: No more than 5 vulnerabilities
 Code Smells: No more than 50 code smells
Duplication: No more than 5% duplicated code
3. Integrate SonarQube into your build process
To perform code analysis and apply the defined quality gate, you
must integrate SonarQube into your build process. You can use build
tools like Maven, Gradle, or Jenkins to achieve this.
For example, if you are using Maven, you can add the SonarQube
plugin to your [Link]:
<plugins>
<!-- Other plugins... -->
<plugin>
<groupId>[Link]</groupId>
<artifactId>sonar-maven-plugin</artifactId>
<version>3.11.0.1612</version>
</plugin>
</plugins>
4. Perform code analysis and quality gate check
After integrating SonarQube into your build process, run the analysis
on your codebase. This can be done using the following Maven
command:
mvn clean verify sonar:sonar
This command will compile your code, run the tests, and execute the
SonarQube analysis. The SonarQube server will receive the analysis
results and evaluate whether the code meets the defined quality gate
conditions.
5. Quality gate results
 Once the analysis is completed, you can navigate to the SonarQube
web interface and check the quality gate results. If the code complies
with all the quality gate conditions, it will pass the quality gate, and
the build process can proceed to the next phase. If the code does not
meet the conditions, it will fail the quality gate, and the issues need
to be addressed before progressing. The following is the sample
SonarQube web screen after the scan (Figure 7.2). We can check
code coverage, reliability, and maintainability of the code in
SonarQube.

Figure 7.2: SonarQube Scan

Likewise, we can check the issue and its resolution under the Issues tab as
shown in Figure 7.3:
 Figure 7.3: Issues in new code under SonarQube Scan

Jenkins pipeline to perform SonarQube analysis

Creating a Jenkins pipeline to perform the SonarQube analysis and
implement the quality gates involves several stages. The following is an
example (Example 7.1) format of a Jenkins declarative pipeline that
demonstrates the process:
pipeline {
agent any
stages {
stage('Checkout') {
steps {
// Checkout your source code repository
// Example: git '[Link]
git'
}
}
 stage('Build and Test') {
steps {
// Build your code and run tests
// Example: sh 'mvn clean verify'
}
}
stage('SonarQube Analysis') {
steps {
// Run SonarQube scanner to perform code analysis
script {
def scannerHome = tool 'SonarQubeScanner' // Assumesn
you have SonarQube scanner configured in Jenkins
def mavenHome = tool 'Maven' // Assumes you have Ma
ven configured in Jenkins
withEnv(["PATH+MAVEN=${mavenHome}/bin"]) {
sh "${scannerHome}/bin/sonar-scanner -[Link]
ctKey=my-project –
[Link]=./src -[Link]=[Link]
}
}
}
}
stage('Quality Gate Check') {
steps {
// Evaluate the quality gate status
 script {
def qualityGateStatus = waitForQualityGate() // Assumes
you have SonarQube
plugin installed in Jenkins
if (qualityGateStatus != 'OK') {
error "Quality gate failed. Fix the issues and try agai
n."
}
}
}
}
}
post {
always {
// Clean up resources or perform any necessary post-build action
s
}
}

}
Example 7.1: SonarQube analysis in Jenkins pipeline

In the preceding Jenkins pipeline, the pipeline has four stages:

1. Checkout: This stage checks out the source code repository where
your Java project is hosted.
2. Build and test: This stage builds your code and runs the tests. You
can customize this step according to your build tool and project
structure.
 3. SonarQube analysis: In this stage, the SonarQube scanner performs
code analysis. You need to have the SonarQube scanner installed and
configured in Jenkins.
4. Quality gate check: The pipeline waits for SonarQube's quality gate
status to determine whether the code meets the quality gate
conditions. If the quality gate fails, the pipeline will be marked as
failed.

Implementing security with the quality gate

method
We will introduce, define, and apply a significantly simplified method to
integrate security as a transversal issue. In the following, we will assume
that any cross-sectional topic can be implemented using the quality gate
methodology. Additionally, the temporal component can be applied to any
cyclical project management methodology. Incorporating this strategy into
the DevSecOps project organization methodology is a perfect fit.
There are different phases in the DevOps process. The various phases are all
seamlessly interconnected. Installing something that interferes with the
entire process at these points is absurd. However, there are better places to
find cross-cutting issues. We are referring to the automated process
derivation found in a CI route. Assuming that the required process steps to
pass through a quality gate can be fully automated, a CI route is perfect for
performing this routine work.
Assuming that the CI line executes an automated process step, two results
can occur:
• Green: Quality gate has passed
This processing step could result in all checks passing correctly. At
this point, processing can continue without interruption. Only a few
log entries are generated to ensure thorough documentation.
• Red: Failed the quality gate
 Another possible result is that the check found something that
indicates a failure. This interrupts the process, and the cause of the
failure, as well as a procedure to correct it, must be found. At this
point, the automatic process normally comes to a stop and is
replaced by a manual process.

Risk management in quality gates

Identifying a defect closes the quality gate, and hence, someone must be in
charge of risk assessment (risk identification, analysis, assessment, and
prioritization), design, and initiation of countermeasures.
The risk determination was already finished with the formation of the
criteria and their operationalization by weighing the requirements on a risk
basis. This occurs during the gate review.
The conception and initiation of countermeasures is a vital activity of a gate
review, at least if the project is not postponed or cancelled before moving to
production. The countermeasures that will be implemented will primarily
address the risks posed by unmet criteria.
Risk management countermeasures are classified as either preventive or
emergency measures. Meeting the criteria as soon as possible is one of the
preventive actions. If this is not possible and practical, adequate
countermeasures must be created. The design of countermeasures is a
creative act that is dependent on the risk, its assessment, and the possible
alternatives.
The effectiveness of the countermeasures must be tracked to ensure their
success. This applies to all phases of the project and is crucial for detecting
and addressing security issues early in the process.

Conclusion
With the increase in supply chain threats in the industry, addressing security
is more important than ever for the success of your project and business. We
must understand the importance of software security and its major
components. When we understand security, we understand that the easiest
way to swiftly minimize vulnerabilities is to shift left and begin addressing
security as a priority concern. Effective software security reduces the risk of
data breaches, unauthorized access, and service disruptions. As cyber threats
continue to evolve, software security must be an ongoing and adaptive
process, encompassing both proactive measures and responsive actions.
This chapter introduced you to the fundamentals of security, including
several analysis methodologies such as SAST, DAST, IAST, and RASP. We
discussed the importance of DevSecOps and how to plan a DevSecOps
pipelines. We also learned about fundamental scoring systems such as the
CVSS. With this knowledge, we will be able to put the appropriate quality
gates and criteria to increase the security of any future projects you work on.
Finally, we discussed practical applications of quality management with
SonarQube and built a Jenkins pipeline to do SonarQube analysis.
In the next chapter, we will discuss about deployment, how to plan the
deployment, building and pushing container images and kubernetes
deployments.

1. Source: [Link]

Join our book’s Discord space

Join the book’s Discord Workspace for Latest updates, Offers, Tech
happenings around the world, New Release and Sessions with the Authors:
[Link]
 CHAPTER 8
Deployment Strategies

Introduction
In the era when computers were massive and prohibitively expensive,
hardware and software were typically bundled together by manufacturers.
However, as mass-market software emerged, this approach became
cumbersome, leading to new distribution methods. Modern development
now prioritizes separating build and deployment tasks to facilitate quick
distribution and parallel work. Deploying an application transforms it from
a packaged artifact into a functional system, and speed is critical to assess
its status and get rapid feedback. As a developer, you must write high-
performance code and ensure it integrates seamlessly with the infrastructure
by clearly communicating deployment instructions to your DevOps team.
During deployment planning, it is beneficial to create a Wish list for future
scalability and efficiency. This Wish list may include implementing frequent
small deployments to gradually enhance system functionalities and enable
easy rollback in case of failures. This ensures isolated Microservice
deployment for scalable and replaceable individual components, promoting
the reusability of deployed Microservices across diverse environments, and
automating infrastructure deployment to seamlessly align with evolving
application features.
Regardless of the container orchestration platform you choose for deploying
your Microservices, the process typically involves packaging the application
and progressing as follows:
• Building and pushing a container image.
• Selecting and implementing a deployment strategy.
As the application deployment progresses across different stages or
environments, you may also be tasked with workload management,
involving refining health checks and optimizing CPU and memory usage to
prevent sluggish or unresponsive functionality. Additionally, addressing
observability aspects becomes crucial, utilizing metrics, logs, and traces to
gain insight into the internals of your distributed systems and measure their
performance.
This chapter guides us through these activities and explores their
implications at scale, emphasizing the importance of a well-structured and
automated deployment process in modern development practices.

Structure
In this chapter, we will discuss following topics:
• Planning the deployment
• Building and pushing container images
• Kubernetes deployments
• Managing workloads in Kubernetes
• Best practices for logging and monitoring
• High availability and geographically distributes system
• Hybrid and multi cloud architecture

Objectives
This chapter aims to equip us with comprehensive skills in deploying and
managing applications in Kubernetes environments. It will cover planning
deployment strategies for both traditional and Kubernetes setups,
highlighting differences and advantages of Kubernetes deployments. We
will learn to efficiently build and push container images using tools like JIB
and Eclipse JKube. Moreover, the chapter will guide us through Kubernetes
deployments, from local setup to choosing and implementing deployment
strategies such as recreate, rolling update, blue-green, and canary
deployments. Additionally, we will discover best practices for workload
management, logging, and monitoring in Kubernetes, ensuring optimal
application performance and system health. Furthermore, we will explore
strategies for achieving high availability and geographically distributed
systems, including hybrid and multi-cloud architectures, providing us with
the skills to navigate complex deployment landscapes in future Kubernetes
projects.

Planning the deployment

Deployment marks the transition of your software from a development or
staging environment to a production environment, where it becomes
accessible to end-users. Proper planning ensures that this transition is
seamless, reliable, and minimizes downtime.
In today's technology landscape, deployment encompasses both traditional
methods and cloud-native approaches like Kubernetes.

Traditional deployment
In a traditional deployment, applications are typically hosted on physical
servers, virtual machines, or cloud instances. The deployment process
typically involves steps like:
1. Code compilation: Compile your source code into executable
binaries or packages.
2. Build artifacts: Create installation packages or container images.
3. Server configuration: Configure the target server environment,
including software libraries and dependencies.
 4. Application installation: Deploy the application to the server.
5. Configuration management: Set up application-specific
configurations.
6. Testing: Conduct thorough testing to ensure the application
functions as expected.
7. Traffic routing: Update DNS or load balancer settings to route
traffic to the new deployment.
8. Monitoring: Implement monitoring and alerting to detect and
respond to issues.

Kubernetes deployment
Kubernetes, an open-source container orchestration platform, revolutionizes
deployment by abstracting infrastructure details and providing tools for
automating the process. The key Kubernetes deployment resources include:
• Deployment: Defines how many replicas of an application should
run and manages their lifecycle.
• Service: Exposes a set of pods as a network service to enable load
balancing and discovery.
• Ingress: Manages external access to services within the cluster.
• ConfigMaps and Secrets: Store configuration data securely and
make it available to pods.

Planning the deployment

Regardless of the deployment method, thorough planning is essential for
success. Here are key considerations:
1. Define objectives and scope
• Goals: Clearly define the objectives of the deployment, such as
improving performance, adding new features, or fixing critical
bugs.
 • Scope: Identify the specific components or services that will be
affected by the deployment.
2. Version control
• Source Code: Ensure that your source code is version-
controlled using tools like Git. Tag releases for easy tracking.
• Infrastructure as Code: If using Kubernetes, store infrastructure
configurations in version-controlled repositories.
3. Testing
• Unit and integration testing: Automate testing processes to
catch issues early.
• Staging environment: Set up a staging environment that mirrors
the production environment for thorough testing.
4. Rollback plan
• Develop a clear rollback plan to quickly revert to a previous
version in case of deployment issues.
5. Monitoring and alerting
• Implement monitoring to track the health and performance of
your application.
• Configure alerting to notify the team of anomalies or failures.
6. Documentation
• Create comprehensive documentation covering the deployment
process, including dependencies, configurations, and
troubleshooting steps.
7. Security
• Implement security best practices, including vulnerability
scanning and penetration testing.
• Protect sensitive data and credentials using encryption and
secrets management.
8. User communication
 • Communicate the deployment schedule to users and
stakeholders to manage expectations.
9. Load balancing
• In Kubernetes, ensure that services are exposed via load
balancers or ingresses for traffic distribution.
Effective deployment planning is a critical step in ensuring the success and
reliability of your software application. By carefully considering your
objectives, scope, and the specific needs of your application, you can
execute deployments with confidence and minimize disruptions to your
users.

Building and pushing container images

Deploying applications to containers requires creating the Java application
artifacts and building container images. Starting with Docker’s appearance
in 2013, building container images using Docker files became popular. In
Chapter 4, Containerization, we discussed that it is critical to understand
that installing Docker is not required for creating container images or
running containers. It is just a popular and convenient tool for doing so. In
the same way you can package a Java project without using Maven or
Gradle, you can build a container image without using Docker or a
Dockerfile. A Dockerfile is a standardized image format comprising the
base operating system, application artifacts to be added, and required
runtime configurations. Essentially, this file is the blueprint for how your
future container will behave.
In the context of DevOps methodology, effective communication between
application developers and infrastructure engineers is crucial. To facilitate
this, some teams find it best to keep the dockerfiles at the repository root.
Scripts or pipelines can also leverage this central location when
orchestrating container image builds. In addition to writing your Dockerfile,
Java-specific tools like Eclipse JKube and Jib offer valuable options when it
comes to creating container images as a routine part of your build processes.
Managing container images by using JIB
Jib is an open-source Java containerizer created by Google that allows you
to build container images for your Java applications without needing to
write a Dockerfile or interact directly with Docker. It integrates seamlessly
with popular build tools like Maven and Gradle, making container image
management a breeze.
Jib takes advantage of image layering and registry caching to achieve fast,
incremental builds. The tool can create reproducible build images as long as
the inputs remain the same.
To begin using Jib within your Maven project, setting up the authentication
method for the target container registry is a crucial step. Jib simplifies this
process by offering multiple authentication options. Here are some common
methods to configure authentication when using Jib:
• Docker credentials: You can provide Docker credentials directly in
your [Link] file. While this is convenient, it may not be the most
secure method, especially for sensitive credentials.
• System properties: System properties [Link] and
[Link].
• Plug-in configuration: <to> section in the plug-in configuration
with username and password elements.
• Docker configuration file: Jib can use the Docker configuration file
(usually located at ~/.docker/[Link]) to authenticate with
container registries. This method is more secure as it separates
credentials from your project files.
• Google Container Registry (GCR) Authentication: If you are
pushing images to Google Container Registry, Jib can automatically
authenticate using your Google Cloud credentials if you have the
necessary permissions.
• Container registry API key: For non-standard registries or when
you prefer not to use Docker credentials, you can provide an API
 key directly in your [Link]. Be cautious with this approach as API
keys can be sensitive.
• Kubernetes secret: In Kubernetes environments, you can use a
Kubernetes secret to store and manage container registry credentials
securely. Jib can be configured to use these secrets.

Caution: If you are using a specific base image registry, you can set up
its credentials by using the <from> section in the plug-in configuration
or [Link] and [Link] system
properties.

Setting up Jib in your Maven project

To start using Jib for managing container images in your Maven project,
follow these steps:
1. Create a Maven project
If you don't already have a Maven project, you can create one using
the following command:
mvn archetype:generate -DgroupId=[Link] -DartifactId=my-j
ava-app -DarchetypeArtifactId=maven-archetype-quickstart -Dintera
ctiveMode=false
2. Add Jib Configuration to [Link]
Open your project’s [Link] file and add the Jib plugin
configuration. This configuration specifies how Jib should build
your container image.
<build>
<plugins>
<plugin>
<groupId>[Link]</groupId>
<artifactId>jib-maven-plugin</artifactId>
 <version>4.3.0</version>
<configuration>
<from>
<image>adoptopenjdk:11-jre-hotspot</image>
</from>
<to>
<image>my-java-app:latest</image>
</to>
</configuration>
</plugin>
</plugins>
</build>
• <from>: Specifies the base image for your container. In this
example, we’re using the AdoptOpenJDK 11 image.
• <to>: Specifies the name and tag for your application’s image.
You can customize this as needed.
3. Build the container image: Now that you have configured Jib in your
[Link], you can build the container image using the following
Maven command:
mvn compile jib:build
4. Verify the container image
You can verify that the container image was built successfully by
running:
docker images.
To start using Jib within your Gradle project, setting up the authentication
method is a critical initial step. Jib simplifies container image management
but requires proper authentication for secure image publishing. Here are
some common methods to configure authentication when using Jib with
Gradle:
• Docker credentials: You can provide Docker credentials directly in
your Gradle build file (typically [Link]). While convenient,
exercise caution as this method may expose sensitive credentials in
the build file.
• Docker configuration file: Jib can utilize the Docker configuration
file (usually located at ~/.docker/[Link]) for authentication. This
approach enhances security by keeping credentials separate from
project files.
• Gradle properties: Storing credentials in Gradle properties files (for
example, [Link]) can be a secure alternative. These
properties can be referenced in your build script to authenticate with
the container registry.
• Kubernetes secrets: In Kubernetes environments, you can utilize
Kubernetes secrets to securely store and manage container registry
credentials. Jib can be configured to use these secrets for
authentication.
• Service account keys: In cloud environments like Google Cloud
Platform (GCP), you can use service account keys for
authentication. This method is suitable for projects running within
cloud platforms.

Setting up Jib in your Gradle project

Here is a step-by-step guide to setting up Jib in your Gradle project:
1. Create a Gradle project
If you do not already have a Gradle project, you can create one using
the following command:
gradle init --type java-library
2. Add Jib plugin to [Link]
 Open your project's [Link] file and add the Jib plugin. Jib
offers two plugins for Gradle: [Link] and
[Link]-base. The former is more user-friendly
and recommended for most projects.
Add the following lines to your [Link] to apply the Jib plugin:
plugins {
id '[Link]' version '4.3.0'
}
3. Configure Jib in [Link]
Next, configure Jib to define how it should build your container
image. You can specify the base image, target image, and other
details. Here's an example configuration:
jib {
from {
image = 'adoptopenjdk:11-jre-hotspot'
}
to {
image = 'my-java-app:latest'
}
}
• from: Specifies the base image for your container. In this
example, we're using the AdoptOpenJDK 11 image.
• to: Specifies the name and tag for your application's image. You
can customize this as needed.
4. Build the container image
Now that you have configured Jib in your [Link], you can
build the container image using the following Gradle command:
 gradle jib
5. Verify the container image
You can verify that the container image was built successfully by
running:
docker images

Building container images with Eclipse JKube

An alternative tool that a Java developer can use to containerize Java
applications without writing docker files is Eclipse JKube. Eclipse JKube is
an open-source Kubernetes-native toolkit that simplifies building,
deploying, and managing containerized applications in Kubernetes
environments. It provides Maven and Gradle plugins for container image
building, Kubernetes resource generation, and seamless integration with
popular Java frameworks like Spring Boot.
To start using the Eclipse JKube Maven plug-in within your project, please
add the Kubernetes Maven plug-in to your [Link]:
<plugin>
<groupId>[Link]</groupId>
<artifactId>kubernetes-maven-plugin</artifactId>
<version>${[Link]}</version>
</plugin>
Let us add the snippet to the sample Spring Boot application from previous
chapters.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="[Link]
xmlns:xsi="[Link]
xsi:schemaLocation="[Link] [Link]
[Link]/xsd/[Link]">
 <modelVersion>4.0.0</modelVersion>
<!-- Parent POM -->
<parent>
<groupId>[Link]</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.5.4</version> <!-- Specify your desired Spring Boot ver
sion -->
</parent>
<!-- Project Information -->
<groupId>[Link]</groupId>
<artifactId>my-spring-boot-app</artifactId>
<version>0.0.1-SNAPSHOT </version>
<!-- Standalone Properties -->
<name>My_Spring_Boot_Application</name> <!-- Specify your projec
t name -->
<description>A sample Spring Boot application with JKube</descriptio
n> <!-- Specify your project description -->
<!-- Properties -->
<properties>
<[Link]>11</[Link]>
<[Link]>0.10.5</[Link]> <!-- Specify
your desired Spring Native version -->
<[Link]>1.5.1</[Link]> <!-Specify your desired JKub
e version ->
 <[Link]> [Link] </[Link]
stry> <!-- Docker registry URL -->❶
<tag>${[Link]}</</tag> <!-- Specify the desired image tag
-->❷
<[Link]>${[Link]}/${repository}/${pr
[Link]}:${tag}</[Link]> <!-- Specify the JKube generat
or name -->
<!-- Repositories -->

<repository> myuser </repository>❸

</properties>
<!-- Dependencies -->
<dependencies>
<!-- Add your Spring Boot dependencies here -->
<dependency>
<groupId>[Link]</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
</dependencies>
<!-- Build Configuration -->
<build>
<plugins>
<!-- Eclipse JKube Plugin Configuration -->
<plugin>
<groupId>[Link]</groupId>
 <artifactId>kubernetes-maven-plugin</artifactId>
<version>${[Link]}</version>
<configuration>
<from>
<image>${[Link]}/adoptopenjdk:11-jre-hots
pot</image>
</from>
<to>
<image>${[Link]}</image> <!-- Use t
he JKube generator defined in properties -->
</to>
</configuration>
</plugin>
</plugins>
</build>
</project>
Example 8.1: [Link] configuration file for sample Spring Boot project

❶ You can provide a default value for the container registry property and
override it at build time.

❷ You can provide a default value for the tag property and override it at
build time. The default image name will be the project name.

❸ You can provide a default value for the repository property and override
it at build time.
With the JKube plugin configured, you can build the container image by
running the following Maven command:
mvn clean package k8s:build
This command will build the Spring Boot application and create a Docker
container image with the specified configuration.
Depending on the technology stack you are using, JKube employs
opinionated defaults such as base images and manually crafted startup
scripts. In this scenario, It uses the current local Docker build context for
both pulling and pushing container images.
Furthermore, the naming of the image is derived from the combination of
Maven properties, including ${[Link]}, ${repository},
${[Link]}, and ${tag}, resulting in a format like:
[Link]/myuser/My_Spring_Boot_Application:0.0.1-
SNAPSHOT.
However, in order to separate the development part from the operational
side, we will customize these details and override them at build time. By
customizing the property, [Link], you can include a remote
registry, repository, image name, and tag of choice:
<[Link]>
${[Link]}/${repository}/${[Link]}:${tag}
</[Link]>
Now, we can build an image for a remote container registry by using the
following command:
mvn clean package k8s:build -[Link]=[Link] -Drepository
=repo -Dtag=0.0.1
Above command will build your Spring Boot application, create a Docker
image for it, and then push that image to the [Link]/repo Docker repository
with the tag 0.0.1. The resulting Docker image will be available on the Quay
container registry under the specified repository and tag.
If you would like to build and push the image to the remote container
registry, you can use this:
mvn clean package k8s:build k8s:push -[Link]=[Link]
-Drepository=repo -Dtag=0.0.1
This command will:
• Build your Spring Boot application.
• Create a Docker image for your application based on the
configuration in your [Link].
• Push the Docker image to the [Link]/repo repository with the tag
0.0.1 on the Quay container registry.
The Docker image that will be built and pushed to the Quay container
registry will be:
[Link]/repo/ My_Spring_Boot_Application:0.0.1
Eclipse JKube, when using remote container registries, searches for
container image credentials in the following locations:
• Docker configuration: Eclipse JKube will first look for Docker
credentials in your Docker configuration. On most systems, Docker
credentials are stored in a configuration file (for example,
~/.docker/[Link] on Linux). These credentials are used for
authentication when pushing Docker images to a registry.
• Kubernetes secret: If Eclipse JKube does not find Docker
credentials in the Docker configuration, it will search for a
Kubernetes Secret in your active Kubernetes context. The secret
should be named jkube-docker-secret. If found, this secret will be
used for authentication when pushing images to a registry.
• Environment variables: If neither Docker configuration nor a
Kubernetes Secret is found, Eclipse JKube will check for
environment variables. You can set the DOCKER_USERNAME and
DOCKER_PASSWORD environment variables to provide Docker
registry credentials. These environment variables should contain the
username and password for the registry.
• Command-line parameters: Finally, you can also provide Docker
registry credentials directly as command-line parameters when
 running Maven or Gradle commands. For example, you can use the -
[Link] and -[Link] parameters to specify the
credentials.
You can use the same steps to build and push container images using the
Eclipse JKube Kubernetes Gradle plug-in. In that case, you should
configure the plug-in in [Link]:
plugins {
id '[Link].k8s' version '1.5.1'
}
jkube {
kubernetes {
registry {
url = 'your-docker-registry-url'
username = 'your-docker-username'
password = 'your-docker-password'
}
}
}}
Replace your-docker-registry-url, your-docker-username, and your-docker-
password with your Docker registry URL and credentials. Alternatively, you
can configure these values in environment variables or other methods
supported by the Eclipse JKube plugin.
To build and push the container image, you can use the following Gradle
task provided by Eclipse JKube:
./gradlew k8sBuild -[Link]=true -[Link]=tr
ue
 • -[Link]=true instructs Gradle to build the Docker
image.
• -[Link]=true instructs Gradle to push the Docker image
to the specified registry.
This command will trigger the build process and push the container image
to the configured Docker registry.

Kubernetes deployments
Understanding how to create and deploy container images is crucial for
managing containers effectively. When working with distributed systems,
Containers play a key role in achieving deployment independence and can
help insulate your application code from potential failures.
In a distributed system, which often involves multiple Microservices,
managing containers becomes a central concern. This is where orchestration
tools come into play, offering a range of capabilities:
• Declarative system configuration
• Container provisioning and discovery
• Monitoring and crash recovery for your system
• Tools for specifying rules and constraints regarding where and how
containers run.
One of the leading solutions in this space is Kubernetes, an open-source
platform designed to automate the deployment, scaling, and management of
containerized workloads. Kubernetes empowers you to organize your
deployments effectively, allowing the platform to create or remove instances
dynamically based on demand. Additionally, it can intelligently replace and
reschedule containers in case of node failures.
Kubernetes has gained widespread popularity due to its features like
portability and extensibility. This popularity has fostered a vibrant
community and strong vendor support. The platform's proven ability to
handle increasingly complex application categories has also made it a
driving force behind enterprise transitions to hybrid cloud and
Microservices architectures.
Kubernetes empowers you to configure your application deployments for
dynamic scalability. It can automatically scale the number of instances of
your service based on changes in load, reducing instances during low-
demand periods and increasing them as needed. Additionally, Kubernetes
can swiftly replace any failed instances with new ones. For developers
looking to deploy applications on Kubernetes, access to a Kubernetes cluster
is essential. A Kubernetes cluster is composed of a set of nodes that runs
containerized applications, as shown in Figure 8.1:

Figure 8.1: Kubernetes components (as per Kubernetes documentation)

Every Kubernetes cluster comprises at least one worker node, with each
worker node hosting Pods. Within a cluster, namespaces serve as a
mechanism to partition and isolate groups of resources, including Pods.
These Pods are integral components that directly interact with your running
containers, created from container images that were previously built and
pushed to a container registry.
When you engage with Kubernetes, you interact with a set of objects that
undergo validation and acceptance by the system. To work with Kubernetes
objects effectively, you must use the Kubernetes API. In the modern
landscape, there are various tools available to facilitate Kubernetes
deployments, including visual aids, command-line interfaces, or Java
extensions such as Dekorate and JKube, which assist in generating and
deploying Kubernetes manifests.

Local setup for deployment

As a software developer, establishing a productive local development
environment is a standard practice. This setup entails configuring access to a
version control system and installing and configuring several essential
components:
• Java Development Kit (JDK): The JDK is necessary for compiling
and running Java applications.
• Build tools (Maven or Gradle): These tools assist in managing
dependencies and building your project.
• Integrated Development Environment (IDE): Popular choices
include IntelliJ IDEA, Eclipse, or Visual Studio Code, which offer
robust coding and debugging capabilities.
• Database or middleware (Optional): Depending on project
requirements, you may integrate a database or middleware to interact
with your code.
• Container tools: Tools like Docker, Podman, Buildah, Jib, JKube,
or similar ones are essential for building, running, and pushing
container images when working with containerized applications.
• Kubernetes development cluster: For Kubernetes-based projects, a
local development cluster is crucial. Options include minikube, kind
(Kubernetes in Docker), Red Hat CodeReady Containers, Docker
Desktop (with a built-in single-node Kubernetes cluster), or Rancher
Desktop. In cases where local clusters consume too many resources,
remote Kubernetes clusters or provisioned environments like the
Developer Sandbox for Red Hat OpenShift can be considered.
This comprehensive local development environment enables software
developers to work efficiently on application features while ensuring
compatibility and ease of deployment.
Before creating any Kubernetes resources, let us summarize some
Kubernetes concepts:
• Cluster: A Kubernetes cluster is a set of machines, called nodes,
that run containerized applications.
• Namespace: Namespaces are used to logically isolate resources
within a cluster. They help organize and manage applications and
resources.
• User: Interaction with the Kubernetes API requires a form of
authentication managed through users.
• Context: A specific combination that contains a Kubernetes cluster,
a user, and a namespace.
• Kubelet: The main agent that runs on each cluster node and ensures
that containers are running and healthy, according to pod
specifications.
• Deployment: A Deployment is a Kubernetes resource that defines
the desired state of an application. It manages replica sets and
ensures that a specified number of Pod replicas are running at all
times.
• ReplicaSet: Under the hood, deployments use ReplicaSets to
maintain a set number of replica pods. If a pod fails, the ReplicaSet
ensures the desired number of replicas is maintained.
• Service: Services define a stable endpoint for accessing your
application, enabling load balancing and network routing to pods. It
is a way to expose an application having multiple instances in
different pods as a network service.
With these concepts in mind, let us explore how you can generate
Kubernetes objects and subsequently deploy them.

Generate Kubernetes manifests using Dekorate

Dekorate can generate Kubernetes manifests at compile time, using Java
annotations and standard Java framework configuration mechanisms. Table
8.1 shows Dekorate Maven dependencies available for Quarkus, Spring
Boot, or a generic Java project:

Framework Dependency

<dependency>
<groupId>[Link]</groupI
d>
Quarkus <artifactId>quarkus-kubernete
s</artifactId>
<version>3.7.0</version>
</dependency>

<dependency>
<groupId>[Link]</groupI
d>
Spring Boot <artifactId>kubernetes-spring-
starter</artifactId>
<version>3.7.0</version>
</dependency>
Generic Java Application <dependency>
<groupId>[Link]</groupI
d>
<artifactId>kubernetes-annotat
ions</artifactId>
<version>3.7.0</version>
 </dependency>
Table 8.1: Dekorate Maven dependencies

Let us create some Kubernetes resources by adding Dekorate to Example

8.1:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="[Link]
xmlns:xsi="[Link]
xsi:schemaLocation="[Link] [Link]
[Link]/xsd/[Link]">
<modelVersion>4.0.0</modelVersion>
<!-- Parent POM -->
<parent>
<groupId>[Link]</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.5.4</version> <!-- Specify your desired Spring Boot ver
sion -->
</parent>
<!-- Project Information -->
<groupId>[Link]</groupId>
<artifactId>my-spring-boot-app</artifactId>
<version>0.0.1-SNAPSHOT</version>
<!-- Standalone Properties -->
<name>My_Spring_Boot_Application</name> <!-- Specify your projec
t name -->
 <description>A sample Spring Boot application with JKube</descriptio
n> <!-- Specify your project description -->
<!-- Properties -->
<properties>
<[Link]>11</[Link]>
<[Link]>0.10.5</[Link]> <!-- Specify
your desired Spring Native version -->
……………
<[Link]>2.0.0</kubernetes-spring-starter.
version> <!-- Specify the latest version of kubernetes-spring-starter -->
<!-- Repositories -->
<repository>myuser</repository>
</properties>
<!-- Dependencies -->
<dependencies>
<!-- Add your Spring Boot dependencies here -->
………….
<!-- Add Dekorate Maven dependency for Spring Boot -->
<dependency>
<groupId>[Link]</groupId>
<artifactId>dekorate-spring-boot</artifactId>
<version>1.0.0</version> <!-- Specify the desired version -->
</dependency>
</dependencies>
<!-- Build Configuration -->
 <build>
………..
</build>
</project>
Example 8.2: [Link] configuration file adding Dekorate

When you build your application without providing specific configuration,

Dekorate will generate deployment and service resource manifests in the
target/classes/META-INF/dekorate directory of your project. These
generated manifests can then be used for deploying your application to
Kubernetes or OpenShift clusters. This generated Service type is ClusterIP
and makes the application available only within the Kubernetes cluster.
However, if you need to make the service accessible externally through a
cloud provider's load balancer, you can achieve this by configuring a
Service resource with the type LoadBalancer.
When working with Dekorate, you have the flexibility to customize the
generation of Kubernetes resources using two main approaches:
• Specifying configurations in [Link]: To ensure a
clean separation between infrastructure and application code, you
can specify configuration details in the [Link] file.
One common customization is setting
[Link]=LoadBalancer. This configuration
informs Dekorate to generate a Kubernetes Service resource of type
LoadBalancer, allowing external access to your application.
• Adding the @KubernetesApplication: Another approach is to
annotate your application's main class, typically denoted as the
DemoApplication class, with the @KubernetesApplication
annotation. This annotation serves as a directive to Dekorate,
providing instructions on generating Kubernetes resources based on
your application's requirements and deployment context.
After configuring your application as needed, you can generate the
Kubernetes objects by packaging the application using the following Maven
command:
mvn clean package
After packaging the application, you will notice among the other files that
are created, two files named [Link] and [Link] in the
target/classes/META-INF/dekorate directory. Either of the manifests we
discussed can be used to deploy to Kubernetes (Example 8.3):
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
[Link]/vcs-url: "[Link] # Replace wit
h your VCS URL
name: My_Spring_Boot_Application
labels:
[Link]/version: "0.0.1-SNAPSHOT"
[Link]/name: "My_Spring_Boot_Application" # Updated na
me
spec:
containers:
- env:
- name: KUBERNETES_NAMESPACE
valueFrom:
fieldRef:
fieldPath: [Link]
image: "repo/demo:0.0.1-SNAPSHOT"
 imagePullPolicy: IfNotPresent
name: "My_Spring_Boot_Application" # Updated name
ports:
- containerPort: 8080
name: "http"
protocol: TCP
Example 8.3: YAML manifest defining a Kubernetes Deployment

This YAML manifest defines a Kubernetes Deployment resource for a

containerized application named My_Spring_Boot_Application. Here is
what it will do when applied to a Kubernetes cluster:
• Deployment: It creates a Kubernetes Deployment named
My_Spring_Boot_Application. A Deployment is a resource that
manages the deployment of multiple replica Pods in a Kubernetes
cluster.
• Pod template: Within the Deployment, there is a Pod template
specified under the [Link] section. This template is used to
create individual Pods. The template includes:
○ Metadata: Labels and annotations are set to help identify and
categorize the Pods.
○ Containers: It defines a single container named
My_Spring_Boot_Application using a container image from the
repository repo/demo:0.0.1-SNAPSHOT. This container runs a
Spring Boot application, and it listens on port 8080 for HTTP
traffic.
• Replicas: The Deployment specifies that it should maintain 1 replica
of the Pod template. This means it will ensure that one instance of
the application is always running.
• Environment variable: The container within the Pod is configured
with an environment variable named
 KUBERNETES_NAMESPACE. This variable is set to the
namespace of the Pod.
• Image Pull Policy: The container's image pull policy is set to
"IfNotPresent." This means that Kubernetes will only pull the
container image from the repository if it is not already present in the
cluster.
• Service annotations: The Deployment includes annotations
indicating the source code's Version Control System (VCS) URL.
Overall, when this YAML manifest is applied to a Kubernetes cluster, it will
create a Deployment that manages one replica of a Pod running the
My_Spring_Boot_Application container. The application will be accessible
within the cluster on port 8080. You can scale the number of replicas by
modifying the replicas field in the Deployment if needed. Now look at how
the service resource can be defined in YAML (Example 8.4):
apiVersion: v1
kind: Service
metadata:
annotations:
[Link]/vcs-url: "[Link] # Replace wit
h your VCS URL
labels:
[Link]/name: My_Spring_Boot_Application
[Link]/version: 0.0.1-SNAPSHOT
name: My_Spring_Boot_Application_Service
spec:
ports:
- name: http
 port: 80
targetPort: 8080
selector:
[Link]/name: My_Spring_Boot_Application
[Link]/version: 0.0.1-SNAPSHOT
type: LoadBalancer
Example 8.4: YAML manifest defining a Kubernetes Service resource

The preceding YAML defines a Kubernetes Service resource, which serves

the purpose of exposing a deployed application within a Kubernetes cluster
to external traffic. Here is what this YAML will do:
• Metadata and labels: It includes metadata and labels to uniquely
identify and categorize the Service resource.
• Annotations: The [Link]/vcs-url annotation is set to
<<unknown>>, indicating that the version control system (VCS)
URL is not specified. Annotations are used to provide additional
information about resources.
• Service ports: It specifies a single port named "http" with the
following properties:
○ Port: 80
○ TargetPort: 8080
This configuration means that traffic coming to port 80 on this
Service will be forwarded to port 8080 on the pods associated with
this Service. Typically, HTTP traffic is directed from port 80 to the
application's internal port 8080.
• Selector: The Service selects pods based on labels with the specified
criteria. In this case, it selects pods with the following labels:
○ [Link]/name: My_Spring_Boot_Application
○ [Link]/version: 0.0.1-SNAPSHOT
 • The service forwards traffic to the pods that match these labels.
• Service type: The Service type is set to LoadBalancer. This
configuration instructs the Kubernetes cluster to allocate an external
load balancer (if supported by the underlying cloud provider) and
route external traffic to the selected pods based on the provided
labels and ports.
This YAML defines a Kubernetes Service that exposes an application
named My_Spring_Boot_Application with version "0.0.1-SNAPSHOT" to
external traffic on port 80 using a load balancer. It ensures that incoming
HTTP requests are forwarded to the pods that meet the specified label
criteria and have their internal service running on port 8080.
Assuming you have previously logged in a Kubernetes cluster and both
yaml are in Kubernetes_demo.yaml file, you can deploy to it using the
command-line interface:
kubectl apply -f target/classes/META-INF/dekorate/[Link]
As a result, you can access the application using the external IP
(LoadBalancer Ingress) and port by Kubernetes after applying the manifests.

Generate and deploy Kubernetes manifests with

Eclipse JKube
Eclipse JKube can also generate and deploy Kubernetes/OpenShift
manifests at compile time. In addition to creating Kubernetes descriptors
(YAML files), you can adjust the output by using the following:
• Inline configuration within the XML plug-in configuration
• External configuration templates of deployment descriptors
“Building Container Images with Eclipse JKube” explored building
container images with JKube and with Docker daemon integration. We will
reuse the [Link] from “Managing Container Images by Using Jib” to
generate and deploy Kubernetes resources with Eclipse JKube and Jib.
You can follow these steps:
 1. Open a command prompt or terminal.
2. Navigate to the directory where your [Link] file is located.
3. Run the following Maven command to build the project and create
Kubernetes resources:
mvn clean package k8s:build k8s:resource
This Maven command performs the following actions:
clean: Cleans the project.
Package: Packages the application.
K8s:build: Builds the container image using Jib.
K8s:resource: Generates Kubernetes resources (Deployment and
Service) using Eclipse Jkube.
4. Once the command completes successfully, you should see output
indicating that the Kubernetes resources have been generated.
5. To apply the generated resources to your Kubernetes cluster, you can
use the kubectl command. For example:
kubectl apply -f target/classes/META-INF/jkube/ubernetes/deploym
[Link]
kubectl apply -f target/classes/META-INF/jkube/ubernetes/service.y
ml
Make sure to adjust the paths to the generated YAML files if they are
located in a different directory.
This will deploy your Spring Boot application as a Kubernetes Deployment
and expose it as a Service.
Please note that you need a running Kubernetes cluster and kubectl
configured to point to the cluster for these commands to work. Additionally,
ensure that Docker is installed and running for the Jib container image build
process.

Choose and implement a deployment strategy

Deploying a single application on Kubernetes can be a straightforward
process, especially when leveraging the appropriate tools. However, as
developers, it is essential to anticipate and plan for the seamless replacement
of an older version of a Microservice with a newer one, all without causing
any downtime.
When deciding on a deployment strategy for Kubernetes, certain quotas and
considerations come into play, including:
• The desired number of instances for your application.
• The minimum threshold for the number of healthy running instances.
• The maximum allowable instances.
Ideally, the goal is to achieve the desired number of running instances as
quickly as possible while utilizing minimal computing resources such as
CPU and memory. To achieve this objective, let us explore some well-
established methodologies and compare their performance.
All-in-one deployment using the recreate strategy is the simplest available
when using Kubernetes Deployment objects as shown below (Example 8.5):
apiVersion: apps/v1
kind: Deployment
metadata:
name: My_Spring_Boot_Application
spec:
replicas: 4
selector:
matchLabels:
app: My_Spring_Boot_Application
strategy:
 type: Recreate❶

revisionHistoryLimit: 15❷
template:
metadata:
labels:
app: My_Spring_Boot_Application
spec:
containers:
- image: repo/demo:0.0.1-SNAPSHOT
name: My_Spring_Boot_Application
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 8080
protocol: TCP
Example 8.5: All-in-one deployment using the Recreate deployment strategy

❶ The deployment strategy is set to "Recreate," which means when

updates are applied, it will destroy existing pods and create new ones. In
this setup, there is no need to define both minimum and maximum instance
limits; only the desired number of instances (in this case, 4) needs to be
specified.

❷ revisionHistoryLimit: 15. So it keeps a revision history of up to 15

revisions.
You can find out the previous revisions by running the following command:
kubectl rollout history deployment/demo
Roll back to a previous version by using the following:
kubectl rollout undo deployment/demo --to-revision=[revision-number]
Although this strategy is efficient in terms of memory and amount of CPU
consumption, it introduces a gap in time when the Microservice is
unavailable.
Another Kubernetes built-in strategy is RollingUpdate, where the current
running instances are slowly replaced by the new ones as shown in Example
8.6:
apiVersion: apps/v1
kind: Deployment
metadata:
name: My_Spring_Boot_Application
spec:
selector:
matchLabels:
app: My_Spring_Boot_Application
strategy:

type: RollingUpdate❶
rollingUpdate:

maxUnavailable: 1❷

maxSurge: 3❸

replicas: 4❹
revisionHistoryLimit: 15
template:
 metadata:
labels:
app: My_Spring_Boot_Application
spec:
containers:
- image: repo/demo:0.0.1-SNAPSHOT
name: My_Spring_Boot_Application
ports:
- containerPort: 8080
name: http
protocol: TCP
Example 8.6: Rolling Update deployment strategy

❶ This Deployment uses the "RollingUpdate" strategy for updates. This

strategy is designed to ensure that updates are performed gradually,
maintaining high availability. It allows for controlled rollouts and rollbacks
of application updates.

❷ maxUnavailable: 1 ensures that during updates, no more than one pod

can be unavailable at a time, minimizing disruption.

❸ maxSurge: 3 allows the Deployment to create up to three new pods

before scaling down the old ones during updates, helping to speed up the
update process.

❹ replicas: 4 here is the Desired number of Pods

This rolling update strategy ensures that the application update is performed
gradually and safely. It maintains a balance between ensuring high
availability (by limiting unavailability to one pod at a time) and speeding up
the update process (by allowing a controlled surge in new pods). It is a
common strategy used to update applications in a Kubernetes cluster
without causing service disruptions.
By focusing on the maximum number of the unavailable Pods, this strategy
safely upgrades your deployment, without experiencing any downtime. But
Applications with complex dependency chains or interdependent
Microservices may face challenges with rolling updates. Ensuring that all
components are updated in the correct order can be complicated and might
lead to extended update times. Also, for applications with a large number of
replicas or a slow start-up time, rolling updates can be time-consuming. If
the update window is too long, it might not meet availability requirements.
Applications with complex start-up or shutdown procedures might not fit
well with the rolling update strategy. The strategy assumes that new pods
can safely replace old ones, which may not always be the case for all
applications. Rolling updates can be challenging for stateful applications
that rely on persistent data. These applications often require careful
coordination to ensure data consistency during updates. In such cases, other
deployment strategies like Blue-Green or Canary deployments might be
more appropriate.
Blue/Green deployment is a release management strategy for deploying
software updates. In this approach, two identical environments (blue and
green) are set up, with one representing the currently running version (blue)
and the other the new version (green) of your application. Initially, all user
traffic is directed to the blue environment, while the green environment
remains idle. During the deployment process, the new version of the
application is deployed and thoroughly tested in the green environment.
Once it is verified to be functioning correctly, a traffic switch is made,
directing users to the green environment, effectively making it the new
production version (green). This approach allows for seamless rollback if
any issues are encountered, as the blue environment is still available.
Blue/Green deployments (Figure 8.2) minimize downtime and reduce the
risk associated with deploying new software versions.
 Figure 8.2: Blue/green strategy

Let us see how we can implement blue/green deployments by using

standard Kubernetes objects:
1. To deploy the blue version (Example 8.7) of the Microservice, which
is labeled with version: blue, we will follow the convention of
associating the label value blue with the blue deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: my-app
version: blue
name: my-app-blue
spec:
replicas: 3 # Number of replicas for the blue version
selector:
matchLabels:
app: my-app
version: blue # Label indicating the blue version
template:
 metadata:
labels:
app: my-app
version: blue # Label indicating the blue version
spec:
containers:
- name: my-app
image: repo/demo:0.0.1-SNAPSHOT # Image for the blu
e version
imagePullPolicy: IfNotPresent # Use the specified image
PullPolicy
ports:
- containerPort: 8080
Example 8.7: Blue Version of Blue/green deployment strategy

Now, expose this deployment by using a Kubernetes Service. After

this, traffic is served from the blue version:

kubectl expose deployment my-app-blue --type=LoadBalancer --name=my-

app-blue-service

2. Apply the green deployment (Example 8.8) of a Microservice having

the label version: green:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: my-app
 version: green
name: my-app-green
spec:
replicas: 3 # Number of replicas for the green version
selector:
matchLabels:
app: my-app
version: green # Label indicating the green version
template:
metadata:
labels:
app: my-app
version: green # Label indicating the green version
spec:
containers:
- name: my-app
image: repo/demo:0.0.1-SNAPSHOT # Image for the gre
en version
imagePullPolicy: IfNotPresent # Use the specified image
PullPolicy
ports:
- containerPort: 8080
Example 8.8: green Version of Blue/green deployment strategy

Switch the traffic from blue deployment to green by patching the Service
object:
kubectl patch svc my-app-service -p '{"spec":{"selector":{"version":"gree
n"}}}'
If the blue deployment is no longer needed, you can remove it by using
kubectl delete.
Although this deployment strategy is more complex and demand additional
resources, it has the advantage of reducing the time between software
development and receiving user feedback. This approach is less disruptive
when it is come to experimenting with new features. If an issue arises with
the green deployment, you can quickly revert to the blue deployment,
ensuring minimal impact on users. This approach enhances system
reliability and minimizes the risk of downtime during updates or releases.
The last strategy that we will look at is a Canary deployment (Figure 8.3).
The idea behind Canary deployment is to release the new version of the
software incrementally and selectively, just like using a canary in a coal
mine to detect toxic gases – if the canary (or a small subset of users) is
unharmed, it is safe to proceed with the broader release.

Figure 8.3: Canary deployment

In a typical Canary deployment, a small percentage of users or servers, often

referred to as "canaries," are chosen to receive the new version of the
application. The rest of the users or servers continue to use the older, stable
version. This approach allows for real-world testing of the new code in a
controlled environment, helping to catch any unexpected issues or bugs
before they impact the entire user base. Monitoring and observability tools
are crucial in canary deployments to closely watch the behavior and
performance of the canaries.
If the canaries perform well and show no signs of problems, the deployment
can gradually expand to a larger audience, and more users or servers start
using the new version. This incremental rollout continues until the entire
user base is on the new version. Canary deployments provide a safety net
for software releases, reducing the blast radius of potential issues and
allowing for quick mitigation if problems arise. They are particularly
valuable in complex, distributed systems, where unforeseen interactions
between services can lead to unexpected problems during deployments.
If you still struggle to choose a deployment mechanism, check out Table
8.2, which summarizes characteristics of the previously discussed strategies:

Rolling Blue/Gree
Recreate Canary
Characteri Update n
Deployme Deployme
stic Deployme Deployme
nt nt
nt nt
Deployme All-at-once Incrementa Parallel Incrementa
nt Strategy l l
Traffic Abrupt Gradual Parallel Gradual
Control cutover transition environme transition
nts
Rollback Limited Good Excellent Good
Capabilitie
s
Resource Least Efficient Efficient Efficient
Efficiency efficient
Rollback Entire Affected Affected Affected
Scope deploymen Pods Pods Pods
t
Complexit Low Moderate High Moderate
 y
Downtime During Minimal Minimal Minimal
deploymen
t
Deployme Fast Moderate Moderate Moderate
nt Speed
Risk Limited Partial High High
Mitigation
Use Cases Small Stateless Complex Microservi
apps, static services apps, ces
dynamic
Observabil Challengin Easier Easier Easier
ity g
Table 8.2: Deployment strategies

Managing workloads in Kubernetes

An application running on Kubernetes is a workload. In the cluster, your
workload will run on one or several Pods having a defined lifecycle.
Managing workloads in Kubernetes involves deploying, scaling, updating,
and monitoring containerized applications. Kubernetes provides various
resources to define and control these workloads, such as Deployments,
StatefulSets, and DaemonSets. Let us explore some best practices for
workload management:
• Use deployments for stateless services: Deployments are ideal for
managing stateless services like web applications. They provide
features like rolling updates, scaling, and self-healing. Define your
application's desired state in a Deployment manifest, and Kubernetes
will ensure that the specified number of replicas is running.
• StatefulSets for stateful services: For stateful services like databases,
StatefulSets are a better choice. They provide stable network
identities and persistent storage. StatefulSets ensure that pods are
 started in a predictable order and maintain their identity across
rescheduling.
• DaemonSets for cluster-wide services: DaemonSets are suitable for
running a copy of a pod on every node in the cluster. This is useful
for cluster-wide services like monitoring agents or log collectors.
DaemonSets ensure that pods are distributed across all nodes.
• Horizontal Pod Autoscaling: Implement Horizontal Pod Autoscaling
(HPA) to automatically adjust the number of pod replicas based on
resource usage metrics like CPU and memory. HPA helps your
application handle varying workloads efficiently.
Previously, we created and applied Kubernetes manifests that included a
Deployment specification. This approach is commonly used for stateless
Microservices. But how can we prevent failure for those Microservices that
depend on an external service or persist their data in a database?
Additionally, as a Microservices codebase evolves, ensuring equitable
utilization of memory and CPU resources becomes important. How can we
effectively handle this?

Setting up health checks

Health checks in Kubernetes are used to determine the state of a container
within a pod. They help Kubernetes orchestrate the deployment and scaling
of applications by ensuring that only healthy containers receive traffic and
that unhealthy containers are automatically restarted or replaced.
Kubernetes supports three main types of Healthchecks:
• Liveness probe: Determines if a container is running as expected. If
the liveness probe fails, Kubernetes will restart the container.
• Readiness probe: Checks if a container is ready to serve traffic.
Containers that fail the readiness probe are temporarily removed
from service, allowing the application to gracefully handle changes
in its state.
 • Startup probe: Used to determine when a container is ready to
receive traffic. It is run only once, during the initial startup of the
container.
Now, let us look at various types of probes:
• HTTP probes: HTTP probes make an HTTP GET request to a
specified endpoint and check for a specific response code within a
given timeframe. This is useful for applications with RESTful APIs
or web servers.
Example YAML for an HTTP readiness probe:
readinessProbe:
httpGet:
path: /healthz
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
• Command probes: Command probes run a command inside the
container and check the exit status. If the command returns a zero
exit status, the probe is considered successful.
Example YAML for a command liveness probe:
livenessProbe:
exec:
command:
- /bin/[Link]
initialDelaySeconds: 10
periodSeconds: 30
• TCP probes: TCP probes check if a specific port on the container is
open. This is useful for applications that rely on low-level network
 connectivity.
Example YAML for a TCP liveness probe:
livenessProbe:
tcpSocket:
port: 8080
initialDelaySeconds: 15
periodSeconds: 20
• gRPC probes: gRPC probes are similar to HTTP probes but
specifically designed for gRPC services. They send a gRPC request
and check for a valid response.
Example YAML for a gRPC readiness probe:
readinessProbe:
grpc:
serviceName: mygrpcservice
portName: grpc-port
initialDelaySeconds: 5
periodSeconds: 10
• Named port probes: Named port probes allow you to specify a
service and port name as targets for the probe. This is useful when
dealing with services that expose multiple ports.
Example YAML for a named port readiness probe:
readinessProbe:
httpGet:
path: /healthz
port: my-port-name
initialDelaySeconds: 5
 periodSeconds: 10
Now, let us see how liveness, readiness and Stratup probes can be integrated
into a Kubernetes deployment manifest (Example 8.9):
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-app
spec:
replicas: 3
selector:
matchLabels:
app: my-app
template:
metadata:
labels:
app: my-app
spec:
containers:
- name: my-container
image: my-image:latest
ports:
- containerPort: 8080
readinessProbe:
httpGet:
 path: /healthz
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 10
failureThreshold: 3
livenessProbe:
tcpSocket:
port: 8080
initialDelaySeconds: 15
periodSeconds: 20
timeoutSeconds: 10
failureThreshold: 3
startupProbe:
exec:
command:
- /bin/[Link]
periodSeconds: 10
timeoutSeconds: 10
Example 8.9: Kubernetes Probes

In the preceding manifest, we have defined a deployment with three pods,

each running a container. We have configured readiness, liveness, and
startup probes using different types, including HTTP, TCP, and command
[Link] uses readiness and liveness probes to determine when the
application is ready to serve traffic and to monitor its ongoing health. The
startup probe is used to check the initial readiness of the container during
startup.
To apply this manifest to your Kubernetes cluster, you can use the following
command:
kubectl apply -f [Link]
Now, see the details of how this Yaml will work for different probes:
• readinessProbe: Configures a readiness probe for the container.
○ httpGet: Performs an HTTP GET request to "/healthz" on port
8080 to check readiness.
○ initialDelaySeconds: Specifies that the probe should start 5
seconds after the container starts.
○ periodSeconds: Sets the probe's frequency to every 10 seconds.
○ timeoutSeconds: Sets the timeout for the probe to 10 seconds.
○ failureThreshold: Defines that after 3 consecutive probe
failures, the container is considered not ready.
• livenessProbe: Configures a liveness probe for the container.
○ tcpSocket: Checks liveness by attempting to establish a TCP
connection to port 8080.
○ initialDelaySeconds: Specifies that the probe should start 15
seconds after the container starts.
○ periodSeconds: Sets the probe's frequency to every 20 seconds.
○ timeoutSeconds: Sets the timeout for the probe to 10 seconds.
○ failureThreshold: Defines that after 3 consecutive probe
failures, the container is considered not live.
• startupProbe: Defines a startup probe for the container.
○ exec: Runs the command /bin/[Link] to determine
when the container is ready.
○ periodSeconds: Sets the probe's frequency to every 10 seconds.
 ○ timeoutSeconds: Sets the timeout for the probe to 10 seconds.

Adjusting resource quotas

In a shared Kubernetes cluster, multiple users or teams may deploy their
applications on a fixed number of nodes. To ensure fair resource allocation
among different applications, cluster administrators use a Kubernetes object
called ResourceQuota. This object sets constraints that limit resource
consumption for a specific namespace (a virtual cluster within Kubernetes).
Resource quotas are typically defined using two primary metrics: CPU and
memory. When setting up resource quotas, you specify two essential
parameters:
• Resource requests: These define the minimum number of resources
that a container needs to run. Kubernetes ensures that a container is
allocated at least the specified resources. Requests are used to
guarantee that a container has the necessary resources to function
correctly.
• Resource limits: Limits define the maximum number of resources
that a container can consume. They prevent a container from
exceeding its allocated resources, which is crucial for maintaining
stability within the cluster.

Application configuration level

Managing resource quotas at the application configuration level involves
defining resource requirements and limits within the application's
configuration files or code. This approach provides fine-grained control
over how much resources individual components or services within an
application can utilize.
Let us consider an example with a Spring Boot application:
In your application's configuration file (for example, [Link]),
you can define resource requirements and limits for different components of
your application:
# Frontend service resource requirements and limits
[Link]-requests=0.5
[Link]-requests=256Mi
[Link]-limits=1
[Link]-limits=512Mi
# Backend service resource requirements and limits
[Link]-requests=0.5
[Link]-requests=512Mi
[Link]-limits=1
[Link]-limits=1Gi
You can access these configuration properties in your code and integrate
them into your application's logic, ensuring that each component receives
the required resources.

Kubernetes deployment configuration level

Managing resource quotas at the Kubernetes deployment configuration level
involves specifying resource requirements and limits directly within the
Kubernetes deployment YAML files. This approach allows you to enforce
resource quotas at the cluster level and maintain centralized control over
resource allocation.
Here is an example (Example 8.10) of defining resource requirements and
limits in a Kubernetes deployment YAML:
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
[Link]/vcs-url: "[Link]
 name: My_Spring_Boot_Application
labels:
[Link]/version: "0.0.1-SNAPSHOT"
[Link]/name: "My_Spring_Boot_Application"
spec:
containers:
- env:
- name: KUBERNETES_NAMESPACE
valueFrom:
fieldRef:
fieldPath: [Link]
image: "repo/demo:0.0.1-SNAPSHOT"
imagePullPolicy: IfNotPresent
name: "My_Spring_Boot_Application"
ports:
- containerPort: 8080
name: "http"
protocol: TCP
resources:
limits:
cpu: 200m
memory: 230Mi
requests:
cpu: 100m
 memory: 115Mi
Example 8.10: Resource requirements and limits in Kubernetes Deployment

In this example, resource requests and limits are set for the container within
the deployment. Kubernetes enforces these constraints, ensuring that the
specified resources are allocated to the container.
There are few cautions, you need to keep in mind:
• If a container specifies a memory limit but does not specify a
memory request, Kubernetes will automatically assign a memory
request that matches the limit.
• Similarly, if a container specifies a CPU limit but does not specify a
CPU request, Kubernetes will automatically assign a CPU request
that matches the limit.

Working with persistent data collections

Microservices architecture, known for its ability to build robust and scalable
applications, emphasizes that each Microservice should independently
manage its data. This principle not only encourages autonomy but also
prevents unintended dependencies, enabling individual deployments.
However, handling persistent data collections in this context requires
thoughtful planning and strategic thinking.
If you are using NoSQL databases like CouchDB or MongoDB, you have
flexibility regarding database changes. You can perform data structure
alterations directly from your application code.
When working with standard SQL databases, managing schema changes is
crucial. Tools like Flyway or Liquibase can help you handle schema
modifications seamlessly. These tools assist in generating migration scripts
and keeping track of which scripts have been executed in the database.
When invoked, these migration tools scan available scripts, identify those
that haven't been run on a specific database, and execute them.
When choosing and implementing a deployment strategy, you should pay
attention to the following:
• Ensure that both database schema versions are compatible with the
application versions used during deployment.
• Maintain schema compatibility with the previous working version of
your containerized application.
• Changing a column's data type necessitates converting all values
stored based on the old column definition.
• Renaming a column, table, or view can be backward-incompatible
unless you use triggers or a programmatic migration script.
Separating the deployment of application code from applying migration
scripts allows for independent management of Microservices. Cloud
providers often offer various managed database solutions, eliminating the
need to manage underlying infrastructure. However, consider data security
and management when using managed database services.
The decision of whether to run databases in Kubernetes depends on aligning
Kubernetes management practices with the operational steps required to
maintain databases. Kubernetes community has resolved these challenges by
implementing operators that incorporate logical domain and operational
runbooks for database management. [Link] provides an extensive
list of operators for managing databases in Kubernetes.

Best practices for logging and monitoring

In the realm of IT, the focus is often on making containerized applications
operational. While during development on your local machine, you are the
primary user of your work, in production, your application faces the entire
world. To ensure your application meets the expectations of all your end
users, it is crucial to observe its behaviour over time under varying
conditions and in different environments.
In recent years, Observability has emerged as a pivotal concept in the IT
industry, but chances are you have already been working on making your
Java applications observable. It refers to the ability to measure and
understand the internal state of a system based on the data it generates,
including logs, metrics, and traces. In essence, observability enables
organizations to gain deep insights into their systems' behaviour, facilitating
effective troubleshooting, performance optimization, and proactive issue
resolution. If you have implemented auditing, exception handling, or event
logging, you have already started observing your application's behaviour. To
build observability for a distributed system, you'll likely employ various
tools to implement monitoring, logging, and tracing practices.
Applications and their underlying infrastructure produce valuable metrics,
logs, and traces that are essential for observing a system correctly. As shown
in Figure 8.4, collecting this telemetry data contributes to visualizing the
state of the system and triggers notifications when any part of your system
is underperforming:

Figure 8.4: Gathering metrics, logs, and traces from applications and infrastructure

In the realm of monitoring, alerts play a pivotal role. Alerts are mechanisms
that notify administrators or automated systems when predefined conditions
or thresholds are met or violated. These conditions can range from resource
utilization exceeding limits to specific error patterns in logs or metrics.
When an alert is triggered, notifications serve as an early warning system,
allowing teams to react promptly. By distributing notifications, you can
identify patterns in the normal workflow of the system. These patterns can,
in turn, help automate the recovery mechanism and execute it whenever an
alert is received. Properly configured alerts are essential for maintaining the
reliability of systems.
Observability provides insight into the state of a distributed system,
enabling you to use this information to address faulty states in your
Microservices. Kubernetes includes a built-in self-healing mechanism that
encompasses actions like restarting failed containers, disposing of unhealthy
containers, and not routing traffic to Pods that aren't ready to serve. To
automate the recovery mechanism further, you can extend Kubernetes' self-
healing capabilities using resources like Jobs and DaemonSets. For
example, you can use a DaemonSet to run a node-monitoring daemon on
every worker node, while a Job can create and retry Pods until a specified
number of them successfully terminate.
Observability also plays a key role in measuring the state of the system
when traffic spikes occur. Applications that respond slowly can frustrate end
users, making it essential to investigate how to scale containerized
applications. Autoscaling eliminates the need for manual intervention when
dealing with traffic spikes, automatically adjusting the number of active
resources and instances.
In Kubernetes, the HorizontalPodAutoscaler (HPA) resource is used to
automatically adjust a workload resource like Deployment to match
demand. It responds to increased load by deploying more Pods and scales
down when the load decreases. The HPA uses an algorithm that considers
the ratio of the desired metric value to the current metric value to determine
the required number of replicas.
HPA calculates desiredReplicas based on the ratio of the current metric
value to the desired metric value. If the current metric value exceeds the
desired value, it triggers the scaling process to increase or decrease the
number of replicas accordingly.
Below is the formula for calculating desiredReplicas :
desiredReplicas = ceil[currentReplicas * (currentMetricValue / desired
MetricValue)]
Here is a breakdown of the components:
• currentReplicas: The current number of replicas in the Deployment
or ReplicaSet.
• currentMetricValue: The value of the metric being observed (for
example, CPU utilization).
• desiredMetricValue: The target or desired value for the metric (for
example,80% CPU utilization).
While HPA focuses on adjusting the number of replicas horizontally,
vertical scaling involves adjusting the resource allocation of existing Pods.
In Kubernetes, Vertical Pod Autoscaler (VPA) is a tool that helps with
vertical scaling. It assigns more CPU or memory resources to Pods when
needed, ensuring efficient resource utilization. However, VPA and HPA
should not be used simultaneously to adjust CPU or memory resources, as it
can lead to undefined behavior across your Pods.
Let us look into some monitoring, logging, and tracing recommendations to
understand observability better when deploying, scaling, and maintaining
containerized applications.

Monitoring
You can use monitoring to observe a system in near real-time. Effective
monitoring relies on collecting and analyzing a variety of metrics. Typically,
this practice involves setting up a technical solution that can gather logs and
predefined sets of metrics, as shown in Figure 8.5.
Metrics are numeric values of system properties over time, like maximum
Java heap memory available or the total number of garbage collections that
occurred. Here are some general types of metrics (Table 8.3) commonly
used in monitoring systems:

Metric
Type Description

Counter Represents a cumulative value based on incrementing an

 integer.

Timer Measures both the count of timed events and the total
time of all timed events.

Gauge A single numerical value that can go up and down

arbitrarily.

Histogr Measures the distribution of values in a stream of data.

am
Meter Indicates the rate at which a set of events occurs.

Table 8.3: Types of metrics

These metrics provide valuable insights into system behavior, performance,

and resource utilization, making them essential for effective monitoring.
Monitoring is a crucial practice that allows you to observe a system in near
real-time. It usually entails implementing a technical solution capable of
collecting logs and predefined sets of metrics, as depicted in Figure 8.5:

Figure 8.5: Pulling and fetching metrics

Several widely used Java libraries for handling metrics include MicroProfile
Metrics, Spring Boot Actuator, and Micrometer. To gain a comprehensive
understanding of your system's behavior, you can collect and query these
metrics using tools like Prometheus. This combination of libraries and tools
empowers you to effectively monitor and analyze your application's
performance and behavior.
To provide you with an example (Example 8.11), we will reuse Example 8.1,
expose its metrics under /actuator/prometheus, and send those to
Prometheus by generating the container image and Kubernetes resources
using Eclipse JKube:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="[Link]
xmlns:xsi="[Link]
xsi:schemaLocation="[Link] [Link]
[Link]/xsd/[Link]">
<modelVersion>4.0.0</modelVersion>
<!-- Parent POM -->
<parent>
……..
</parent>
<!-- Project Information -->
…………….
<!-- Standalone Properties -->
<name>My_Spring_Boot_Application</name> <!-- Specify your projec
t name -->
<description>A sample Spring Boot application with JKube</descriptio
n> <!-- Specify your project description -->
<!-- Properties -->
<properties>
…………
</properties>
<!-- Dependencies -->
<dependencies>
 <!-- Add your Spring Boot dependencies here -->
………..
<!-- Add the Micrometer Prometheus registry dependency -->
<dependency>
<groupId>[Link]</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
<scope>runtime</scope>
</dependency>
</dependencies>
<!-- Build Configuration -->
<build>
<plugins>
<!-- Eclipse JKube Plugin Configuration -->
<plugin>
<groupId>[Link]</groupId>
<artifactId>kubernetes-maven-plugin</artifactId>
<version>${[Link]}</version>
<executions>
<execution>
<id>resource</id>
<goals>
<goal>resource</goal>
</goals>
</execution>
 </executions>
<configuration>
<!-- Specify the generator for Kubernetes resources (for e
xample,spring-boot) -->
<generator>
<name>spring-boot</name>
<!-- Expose port 9779 at the container-image level --
>
<ports>
<port>9779</port>
</ports>
</generator>
<!-- Expose the Prometheus port -->
<enricher>
<config>
<jkube-prometheus>
<prometheusPort>9779</prometheusPort>
</jkube-prometheus>
</config>
</enricher>
<!-- Specify the container image name -->
<image>
<name>${[Link]}/${repository}/${pro
[Link]}:${tag}</name>
</image>
 </configuration>
</plugin>
</plugins>
</build>
</project>
Example 8.11: [Link] with Micrometer Prometheus registry dependency

In the context of the Eclipse JKube Maven Plugin for Kubernetes resource
generation, the generator and enricher sections serve specific purposes in the
above yaml:
• Generator section: The generator section in the YAML file defines
the Kubernetes resource generator to use, specifically tailored for
Spring Boot applications. It also specifies port configurations at the
container-image level. For example, in the provided [Link], it sets
the generator as "spring-boot" and exposes port 9779, commonly
used for Prometheus metrics. This allows Prometheus to collect
metrics from the application.
• Enricher section: In the same YAML file, we have an enricher
section. The enricher section configures various enrichers
responsible for enhancing the Kubernetes resource descriptors
generated by JKube. One notable enricher is the jkube-prometheus
enricher, which focuses on Prometheus-related settings. In the
provided [Link], it specifies the jkube-prometheus enricher and
sets the prometheusPort to 9779. This ensures that Kubernetes
resources, such as Services and Deployment objects, correctly
expose port 9779 for Prometheus to scrape metrics.
In summary, the generator section defines the resource generator and
manages container image port exposure, while the enricher section
configures enrichers, like jkube-prometheus, to customize Kubernetes
resource properties for Prometheus support. These sections together
facilitate the creation of Kubernetes resource descriptors suitable for
monitoring Spring Boot applications with Prometheus.
To build the container image and generate the Kubernetes resources, run the
following command:
mvn clean package k8s:build k8s:resource
The Kubernetes resources generated by Eclipse JKube at
target/classes/META-INF/jkube/[Link] will include annotations
that control the metrics collection process for your Spring Boot application
with Prometheus support.
These annotations are used to specify how Prometheus should scrape
metrics from your application. They define which endpoints Prometheus
should target for metric collection and may include other configuration
details.
Here is an example (Example 8.12) of what the annotations controlling
Prometheus metrics collection might look like in the generated Kubernetes
resources:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-spring-boot-app
spec:
template:
metadata:
annotations:
[Link]/scrape: "true" # Enable scraping for this pod
[Link]/port: "9779" # Specify the port for Prometheus to
scrape
[Link]/path: "/actuator/prometheus" # path to the Promet
heus metrics
 spec:
containers:
- name: my-spring-boot-app
ports:
- containerPort: 8080 # Your application's main port
Example 8.12: Prometheus metrics collection

These annotations tell Prometheus to scrape metrics from the specified path
on the container's port 9779, where your Spring Boot application exposes its
Prometheus metrics. You can adjust these annotations based on your
application's configuration and how you want Prometheus to collect metrics.
Once you deploy the generated resources, you can use a custom
Prometheus query (PromQL) to query different metrics. For example, you
can pick the [Link] metric and run the following PromQL query to
check the average time spent in garbage collection by cause:
avg(rate(jvm_gc_pause_seconds_sum[1m])) by (cause)
When generating and capturing metrics in a system, several best practices
should be followed to ensure effective monitoring and observability:
• Collaborative metric definition: Metrics can be defined at both the
application and infrastructure levels. Encourage team collaboration
in defining these metrics to ensure comprehensive coverage.
• Define clear objectives: Start by defining clear objectives for what
you want to monitor and measure in your system. Understand the
key performance indicators (KPIs) that matter most for your
application or service.
• Application-specific metrics: Create metrics for application-
specific implementations that impact nonfunctional requirements.
For example, consider monitoring cache statistics like size, cache
hits, and entry time-to-live. These metrics can provide valuable
insights into functionality performance.
 Expose JVM metrics: Ensure that you expose internal JVM
•
metrics, including data on the number of threads, CPU usage,
garbage collector performance, heap, and non-heap memory usage.
These are essential for understanding your application's
performance.
• Document your metrics: Maintain clear documentation for your
metrics, including their definitions, intended use, and any specific
considerations for interpretation.
• Scale metrics storage: Plan for the scalability of your metrics
storage system as your system grows. Ensure it can handle increased
data volume efficiently.
• Use histograms and summaries: For latency-related metrics,
consider using histograms and summaries to capture and analyze
latency distributions.
• Continuous improvement: Continuously refine your metrics
strategy based on the insights gained from your monitoring and
observability practices.
By following these best practices, you can establish a robust metrics
collection and monitoring framework that provides valuable insights into
the performance and health of your systems.

Logging
Logging involves recording specific events, actions, or information within
an application or system. These records, known as log entries or log
messages, serve as a historical account of what transpired during the
software's execution. Logging can encompass various aspects:
• Error tracking: One of the primary purposes of logging is to
capture errors and exceptions. When an application encounters an
issue, it logs details about the error, including its type, location, and
context. These error logs are invaluable for diagnosing and fixing
problems swiftly.
 • Performance monitoring: Logs can provide insights into the
performance of an application. By recording execution times,
resource consumption, and other performance metrics, developers
can identify bottlenecks and optimize code for efficiency.
• Auditing and compliance: In many industries, applications must
adhere to strict auditing and compliance requirements. Logging
helps in creating an audit trail, ensuring that actions and transactions
are recorded for later review and compliance checks.
In the realm of Java application development, logging serves as a vital tool
for capturing exceptional cases. These logs offer valuable insights enriched
with contextual information, complementing existing metrics. Logging, as a
practice, encompasses three distinct formats: plain text, JSON or XML, and
binary.
When it comes to implementing robust logging in Java applications,
developers can leverage various logging frameworks beyond the language's
built-in options. Notable choices include the Simple Logging Facade for
Java (SLF4J) and Apache Log4j 2. To ensure effective logging, adhering to
best practices is essential:
• Exercise caution: Log only pertinent details related to specific
functionalities within your system. Avoid cluttering logs with
irrelevant information.
• Craft informative messages: Write log messages that convey
meaningful information. This practice aids not only you but also
your colleagues in troubleshooting potential issues in the future.
• Utilize appropriate log levels: Assign log levels judiciously. Use
TRACE for capturing fine-grained insights, DEBUG for
troubleshooting assistance, INFO for general updates, and WARN or
ERROR to signal events demanding immediate attention.
• Conditionally log: Employ guard clauses or lambda expressions to
log messages only when the corresponding log level is active. This
ensures efficient log processing.
 • Customizable log levels: Make log levels configurable via variables
that can be adjusted during container runtime. This flexibility allows
for dynamic log management.
• Secure log locations: Set appropriate permissions for the directories
where log files reside, safeguarding them from unauthorized access.
• Tailored log layouts: Customize log layouts to suit specific regional
or contextual formatting requirements.
• Protect sensitive data: Exercise caution when logging potentially
sensitive information. For instance, avoid logging personally
identifiable information (PII), as doing so may lead to compliance
violations and security vulnerabilities.
• Log rotation: Implement periodic log rotation to prevent log files
from becoming excessively large. For container and Pod logs, which
are transient by default, consider asynchronous streaming to
centralized storage and automated log retention policies. This
ensures that crucial log data is preserved even when Pods are
deleted, crash, or migrate to different nodes.

Tracing
In today's fast-paced digital landscape, ensuring the smooth operation of
complex, distributed software systems is paramount. As applications grow
in complexity, pinpointing and resolving issues swiftly becomes
increasingly challenging. This is where distributed tracing comes to the
rescue, offering developers deep insights into the inner workings of their
applications. In a distributed system, requests traverse multiple components,
making it crucial to capture metadata and timing details. This helps in
identifying slow transactions and pinpointing the sources of failures.
When it comes to selecting the right instrumentation for trace capture,
developers often face challenges. While proprietary agents can assist in this
regard, it is advisable to explore solutions that adhere to vendor-neutral,
open standards like OpenCensus or OpenTracing. Recognizing the
complexity developers encountered in choosing the best option that works
seamlessly across different vendors and projects, the OpenTracing and
OpenCensus projects merged, giving rise to another CNCF incubating
project known as OpenTelemetry.
OpenTelemetry is a comprehensive suite comprising tools, APIs, and SDKs
that standardize the collection and transmission of metrics, logs, and traces.
Within the OpenTelemetry tracing specification, several essential terms are
defined:
• Trace: This represents a single transaction request as it traverses a
distributed system while utilizing various services and resources.
• Span: Spans are named, timed operations that serve as
representations of workflow segments. A trace typically
encompasses multiple spans.
• Attributes: Attributes are key/value pairs that provide valuable
context for querying, filtering, and comprehending trace data.
• Baggage items: These are key/value pairs that can traverse process
boundaries, ensuring continuity of context across different parts of a
system.
• Context propagation: It is a shared subsystem integral to traces,
metrics, and baggage items. Developers can pass additional context
information to a span using attributes, logs, and baggage items,
facilitating comprehensive trace analysis and understanding.
To incorporate distributed tracing into your application, you will need to
follow these steps:
1. Choose a tracing system: There are several popular distributed
tracing solutions available, such as Jaeger, Zipkin, and
OpenTelemetry. Select one that aligns with your technology stack
and requirements.
2. Instrument your code: Integrate tracing libraries or SDKs into your
application's codebase. These libraries are responsible for creating
and managing spans.
 3. Define instrumentation points: Identify the key operations and
interaction points in your code where you want to start and stop
spans. For example, you might want to trace database queries, HTTP
requests, or method calls.
4. Propagation and context: Ensure trace context is properly
propagated across service boundaries. Most tracing systems provide
support for various communication protocols to achieve this.
5. Visualize and analyze: Use trace visualization tools provided by
your chosen tracing system to monitor and analyze the collected
data. This helps in identifying bottlenecks and optimizing your
system's performance.
Here are some recommended tracing practices to enhance your
observability:
• End-to-end instrumentation: Ensure comprehensive trace coverage
by forwarding tracing headers to all downstream services, data
stores, or middleware components within your system. This provides
a holistic view of request flow.
• Report vital metrics: Track crucial metrics related to request rate,
errors, and their durations. Following the Rate, Errors, Duration
(RED) methodology, focus on instrumenting request throughput,
error rates, and latency/response times. These metrics are essential
for effective system monitoring.
• Minimal metadata: If you choose to instrument custom tracing
spans, exercise restraint in adding excessive metadata. While
metadata can provide context, an overabundance may lead to
complexity and reduced trace clarity.
• Java-specific tracing: For Java-based applications, explore
language-specific implementations of OpenTelemetry in Java.
Leveraging Java-compatible tracing solutions can streamline
integration and improve trace visibility.
• Correlate logs and traces: Connect your log data with traces by
including trace and span IDs in log entries. This correlation aids in
 root cause analysis.
• Alerting and monitoring: Set up alerts based on trace data to
proactively identify issues and maintain system reliability.
When designing systems for observability, always keep in mind that your
metrics and logs should be readily accessible for subsequent analysis.
Therefore, regardless of your deployment environment, establish robust
tools and practices for capturing, storing, and retaining metrics and logging
data reliably. This ensures that you have the necessary insights to
troubleshoot and optimize your systems effectively.

High availability and geographically distributed

system
In today's interconnected world, the demand for high availability and
geographically distributed systems has become paramount. Users expect
uninterrupted access to applications and services, regardless of geographical
location or potential disruptions. This article explores the principles,
strategies, and best practices for creating highly available and
geographically distributed systems.
High availability (HA) is a measure of how consistently a system can
deliver its services without interruption. It is typically expressed as a
percentage, indicating the fraction of time the system is operational in a
given period. Table 8.4 illustrates HA percentages and their corresponding
downtime per year:

Availability Percentage Downtime per Year (Approx.)

90% (1 Nine) 36.5 days
99% (2 Nines) 3.65 days
99.9% (3 Nines) 8.76 hours
99.99% (4 Nines) 52.56 minutes
 99.999% (5 Nines) 5.26 minutes
99.9999% (6 Nines) 31.5 seconds
Table 8.4: HA percentages and their corresponding downtime

To establish highly available systems, reliability engineering outlines three

fundamental principles for system design:
• Elimination of single points of failure: This principle advocates the
eradication of vulnerabilities at multiple levels within the system,
including the application, network, and infrastructure layers. It is
essential to comprehensively test each software component to
identify and rectify potential sources of failure. Employing
observability tools and implementing robust deployment strategies
play a pivotal role in mitigating the risk of failures within the
system.
• Real-time failure detection: To ensure high availability, it is
imperative to detect failures as they happen. This necessitates the
implementation of robust monitoring and alerting mechanisms that
continuously monitor the system's health. These tools provide early
detection of critical conditions, enabling proactive responses to
potential issues.
• Seamless transition in the face of failures: In the event of a failure,
a dependable transition to an operational component is crucial. This
can be achieved through an efficient rollback process in cases of
deployment-related problems. Kubernetes, with its self-healing
capabilities, enhances the system's resilience by automatically
replacing failed components. Additionally, smooth traffic routing
mechanisms between Kubernetes resources facilitate uninterrupted
service during component transitions.
A comprehensive strategy for handling failures entails the application of the
aforementioned principles through the implementation of several best
practices:
• Data management: Implement robust data backup, recovery, and
replication procedures to safeguard critical data in case of failures.
 • Network load balancing: Deploy network load balancing
mechanisms to efficiently distribute traffic, particularly during
periods of increased workloads. This approach not only enhances
application performance but also eliminates potential single points of
failure at the application level, utilizing the available network and
infrastructure resources effectively.
• Geographical redundancy: Mitigate the impact of natural disasters
and system disruptions by deploying your applications in multiple
geographical locations. Running independent application stacks in
each location ensures that other locations can continue to operate
even if one experiences a failure. Ideally, these locations should be
distributed globally to avoid localization in a specific area.
• Kubernetes high availability: For Kubernetes clusters, especially
when concerned about performance during component or control-
plane node failures, opt for highly available Kubernetes setups.
These configurations involve multiple control-plane nodes operating
as a unified data centre. Such a setup safeguards against the loss of a
worker node due to a control-plane node's etcd failure. While
managing such clusters can be complex, many cloud providers offer
pre-configured high-availability setups when provisioning clusters.
• Namespace management: Depending on your specific
requirements, setting up multiregional Kubernetes clusters may not
always be justifiable. However, you can still ensure availability by
creating multiple namespaces within the same cluster, effectively
isolating applications and their resources.
Here are some best practices for building and maintaining highly available
and geographically distributed systems:
• Automate everything: Automation reduces the risk of human error
and ensures consistency in managing resources.
• Regular testing: Perform regular testing and simulation of failures
to validate your HA and disaster recovery strategies.
• Security measures: Implement robust security measures to protect
against cyber threats and data breaches.
 • Scalability: Design systems to scale horizontally to accommodate
increased loads.
• Documentation: Maintain comprehensive documentation of system
architecture, configurations, and procedures.
• Continuous improvement: Continuously assess and improve your
system's resilience and availability.
Utilizing the practice of multiregional deployments offers a significant
advantage in enhancing the end-user experience, particularly in maintaining
low latencies for a globally distributed user base. This strategic approach in
your application architecture ensures that data remains in proximity to end
users, effectively reducing latency for users across the world.
Another critical consideration when dealing with geographically distributed
applications is the imperative need to adhere to data privacy laws and
regulations. In today's digital landscape, privacy and data protection have
gained heightened importance. Several countries now enforce stringent
regulations that prohibit the collection, usage, and sharing of personal
information without proper notice or consent from consumers.
Understanding the requirements for ensuring the high availability of a
distributed system sets the stage for exploring cloud models that can
effectively support these objectives. Let's delve into the cloud computing
models that can play a pivotal role in achieving and maintaining high
availability.

Hybrid and multicloud architectures

The cloud is a collection of technologies to approach challenges like
availability, scaling, security, and resilience. It can exist on premises, on a
Kubernetes distribution, or in a public infrastructure. Often, you will see the
terms hybrid cloud and Multicloud used synonymously. The most intuitive
definition for a Multicloud architecture is that this type of architecture
requires at least one public cloud.
In the ever-evolving landscape of cloud computing, hybrid and Multicloud
architectures have emerged as powerful strategies to meet diverse business
needs. These approaches allow organizations to leverage the advantages of
multiple cloud providers and on-premises infrastructure.

Hybrid architecture
Hybrid architecture refers to an IT infrastructure that combines elements of
on-premises and cloud-based solutions. In a hybrid setup, some workloads
and data reside in on-premises data centers, while others are hosted in public
or private cloud environments. This blend of resources provides flexibility,
scalability, and cost-effectiveness.
Key characteristics of hybrid architecture include:
• Data integration: Data can seamlessly move between on-premises
and cloud environments, often facilitated by hybrid cloud solutions.
• Resource flexibility: Organizations can dynamically allocate
resources based on workload demands. This flexibility allows them
to scale up or down as needed.
• Improved disaster recovery: Hybrid setups often include redundant
systems, enhancing disaster recovery capabilities.

Multicloud architecture
Multicloud architecture involves using services and resources from multiple
cloud providers, rather than relying on a single provider. In a Multicloud
strategy, organizations select the most suitable cloud services for specific
workloads or applications, creating a diverse ecosystem.
Key characteristics of Multicloud architecture include:
• Vendor diversity: Organizations can choose the best-fit cloud
provider for each application or service, avoiding vendor lock-in.
• Risk mitigation: Multi-cloud reduces the risk of downtime due to
provider-specific outages. If one provider experiences issues, others
can continue to operate.
 • Cost optimization: By selecting the most cost-effective services
from various providers, organizations can optimize their cloud
spending.
Hybrid cloud architecture differs from Multicloud by including a private
cloud infrastructure component and at least one public cloud (Figure 8.6).
As a result, when a hybrid cloud architecture has more than one public
offering, that architecture can be simultaneously a Multicloud one.

Figure 8.6: Multicloud and hybrid cloud

When embarking on deployments across hybrid or multicloud

infrastructures, it is vital to take into account several cross-team aspects:
• Unified deployment visibility: Ensure that you maintain a unified,
comprehensive view of what you have deployed and where it is
located within the hybrid or multicloud setup. This clarity aids in
efficient management and monitoring.
• Provider-agnostic solutions: Aim to replace provider-specific
Software as a Service (SaaS) and Infrastructure as a Service
(IaaS) services with solutions that are agnostic to cloud providers.
This approach enhances flexibility and reduces vendor lock-in.
• Unified security approach: Implement a unified approach to
address security vulnerabilities consistently across multiple cloud
environments. This ensures a cohesive and robust security posture.
• Seamless scaling and provisioning: Facilitate seamless scaling out
and provisioning of new resources as needed to meet evolving
 demands across different clouds. This agility is crucial for
optimizing resource allocation.
• Ensuring service continuity: When migrating applications across
different cloud environments, it is essential to avoid service
disruptions. While there may be a brief time required for recovery
during workload migration, you can ensure a smooth transition for
end-users through appropriate network configurations and
deployment strategies.
• Automation at scale: Automation plays a pivotal role when
orchestrating processes at such a large scale. In addition to the
orchestration platform for containerized applications, you and your
team may incorporate an additional layer of tools and processes to
efficiently manage workloads.
These considerations are integral to successfully navigating the complexities
of Hybrid and Multicloud deployments while maintaining operational
efficiency and reliability.
From a developer's perspective, actively participating in a hybrid or
Multicloud strategy involves attention to the following key elements:
• Consistent codebase: Ensure that your applications codebase
remains consistent regardless of the environment or namespace it
operates in. This uniformity simplifies management and deployment
across diverse cloud environments.
• Reproducible practices: Establish local building and deployment
practices that are easily reproducible by your colleagues when they
work with your code. This promotes collaboration and consistency
within the development team.
• Avoid local dependencies: Refrain from referencing local
dependencies in your code or during the container image build
process. Instead, favor dependencies that are readily accessible and
well-documented.
• Parameterization: Whenever feasible, parameterize the container
image through build-time variables or environment variables. This
 flexibility allows for dynamic adjustments to suit different
deployment environments.
• Environment customizations: If specific environment
customizations are required, propagate them using environment
variables from the orchestration platform to configure
container/application parameters. This approach maintains flexibility
while minimizing code alterations.
• Trusted repositories: Utilize dependencies and container images
from repositories and registries that your organization has previously
validated as trusted and secure sources. This practice enhances
security and reliability.
• Volume usage: Prefer the use of volumes for sharing information
among containers when necessary. Volumes offer a structured and
efficient way to exchange data between components.
When striving for a hybrid or Multicloud architecture, it is essential to
consistently envision how the software you're building will evolve.
Embracing a forward-thinking developer mindset lays the foundation for a
progressive software architecture that aligns with your organization's
strategic goals.

Conclusion
This chapter discussed the deployment aspects that may be relevant to a
Java developer. We started the chapter with details of planning the
deployment. While the typical responsibilities of a Java developer may not
include infrastructure administration, you can still have a significant impact
on the operational stages and processes of your application by taking the
following actions:
• Building and pushing container images to container image registries
using Java-based tools like Jib and Eclipse JKube.
• Generating and deploying Kubernetes manifests using Dekorate and
Eclipse JKube.
• Different Deployment strategies.
 • Implementing health checks and orchestrating their execution at the
infrastructure level.
• Observing the behavior of the distributed system to determine when
to implement changes and which resources to adjust.
• Linking deployment considerations with high availability, Hybrid,
and Multicloud architectures.
Given your familiarity with deploying applications, the next chapter will
discuss the importance of continuous delivery and its best practices.

Join our book’s Discord space

Join the book’s Discord Workspace for Latest updates, Offers, Tech
happenings around the world, New Release and Sessions with the Authors:
[Link]
 CHAPTER 9
Continuous Delivery and
Deployment

Introduction
In the ever-evolving landscape of software development, the pursuit of
reliable and efficient methods for delivering software is paramount.
Continuous delivery (CD) and Deployment are two interconnected
practices that have redefined the way software is brought from conception to
the hands of end-users. These approaches stand as pillars of agility and
reliability, offering a means to streamline the development process, reduce
manual interventions, and ensure that every software release is not just
functional but also dependable.
In this chapter, we embark on a journey to explore the fundamental
concepts, benefits, and best practices of continuous delivery and
deployment, unveiling their critical role in the quest for dependable software
releases.

Structure
In this chapter, we will discuss following topics:
• Continuous delivery:
 • Continuous uptime
• Continuous delivery and continuous deployment
• CI/CD pipeline example

Objectives
This chapter will explore the evolution of CD as an extension of continuous
integration (CI). It will cover its key components, benefits, and the need
for continuous updates to meet user expectations. The importance of
continuous uptime will also be highlighted, with real-world case studies
demonstrating its role in maintaining uninterrupted services.
The chapter will then examine the relationship between continuous delivery
and continuous deployment. It will outline their significance, best practices,
and challenges. A comparison of the two approaches will guide in choosing
the most suitable method. Additionally, a practical CI/CD pipeline example
will demonstrate how both can be implemented effectively.

Continuous delivery:
In today's fast-paced software development landscape, the demand for rapid
and reliable software delivery has never been higher. CI and CD have
emerged as indispensable practices to meet this demand. While continuous
integration ensures that code changes are integrated and tested frequently,
continuous deployment takes this a step further by automating the
deployment process. Let us delve into continuous delivery, its significance,
and how it is an extension of CI, emphasizing its pivotal role in enhancing
software security.

Continuous integration recap

Before we dive into continuous delivery, let us briefly recap continuous
integration. CI is a software development practice that involves regularly
merging code changes into a shared repository, followed by automated
builds and tests. The primary goal of CI is to detect and address integration
issues early in the development cycle. CI has become the standard practice
in modern software development, ensuring that code changes do not disrupt
the stability of a project. While CI focuses on early error detection and code
integration, it alone is insufficient to guarantee the security of software in
today's complex threat landscape.

The evolution to continuous delivery

Continuous delivery is the natural evolution of CI. While CI focuses on
integrating code changes, CD goes a step further by addressing the
continuous delivery of those changes into various environments, including
production. In other words, CD extends the CI pipeline by automating the
entire release process, from code commit to deployment. While continuous
delivery extends the principles of CI to encompass the entire software
deployment process, It goes beyond mere code integration and automated
testing, introducing additional layers of security measures to address
vulnerabilities that may be exposed during the deployment pipeline.

Key components of continuous delivery

Some key components of CD have been listed as follows:
• Automation: At the heart of CD is automation. The entire software
delivery pipeline, from building and testing to deployment, is
automated (including incorporation of automated security scanning
tools). This minimizes human error, speeds up the process, identify
potential security issues and ensures consistency.
• Testing: Just like CI, CD emphasizes automated testing. Test suites
(including security testing) are executed at various stages of the
pipeline, and only if all tests pass is the code considered for
deployment.
• Environment management: CD involves managing multiple
environments, including development, testing, staging, and
production. Each environment mirrors the production environment
as closely as possible, reducing the risk of unexpected issues during
deployment.
 Version control: CD relies on version control systems to track
• changes, ensuring that the code deployed is the same as what was
tested in earlier stages of the pipeline.
• Deployment strategies: CD introduces various deployment
strategies, such as blue-green deployments and canary releases,
which allow for gradual and controlled releases to production.

Benefits of continuous delivery

Some benefits of CD are as follows:
• Faster time to market: CD reduces the time it takes to get new
features or bug fixes into the hands of users. This speed is essential
in highly competitive industries.
• Higher quality software: The automation and rigorous testing in
CD result in a higher level of software quality and stability.
• Reduced risk: By automating the deployment process and
leveraging testing, CD minimizes the risk of errors and disruptions
in production.
• Improved collaboration: CD encourages collaboration among
development, testing, and operations teams, promoting a DevOps
culture.
• Customer satisfaction: Faster and more reliable releases lead to
happier customers, who benefit from a steady stream of
improvements and bug fixes.
Continuous delivery is a natural progression from continuous integration
and is essential for modern software development. By extending the
principles of CI to encompass the entire software delivery process, CD
provides a systematic and efficient way to bring code changes into
production. Embracing continuous delivery can significantly enhance the
speed, quality, and reliability of software releases, making it a crucial
practice for any development team striving to meet the demands of today's
dynamic markets.
Need for continuous updates
The technological landscape is in a constant state of flux, with new
advancements and innovations arriving at an unprecedented rate. In this
climate, users demand more than static solutions. Continuous updates offer
the agility needed to stay relevant, secure, and competitive. In today's digital
landscape, users have heightened their expectations when it comes to
software and services. Security has become a paramount concern.
The digital world is constantly under siege from evolving and sophisticated
cyber threats. Users now not only expect seamless and efficient software but
also demand that their data and systems remain secure. Continuous updates,
viewed through the lens of security, are the cornerstone of ensuring that
digital products and services stay resilient, safeguarded, and ahead of
potential vulnerabilities. From the lens of security, here are some points
discussing the importance of continuous updates:
• Enhanced security measures: Users increasingly look to software
developers to provide robust security measures as part of their
services. Continuous updates allow developers to reinforce security
protocols, patch vulnerabilities, and implement encryption to
safeguard user data.
• Timely vulnerability fixes: In a world where new vulnerabilities
surface daily, users depend on timely updates to address and rectify
security flaws. Continuous updates serve as the immediate response
to emerging threats, ensuring user data and privacy remain protected.
• Adaptation to emerging threats: Cyber threats are ever evolving,
and software must adapt accordingly. Continuous updates are a
proactive measure to stay ahead of threats, including zero-day
vulnerabilities and malware attacks, mitigating potential damage.
• Competitive advantage: From a business perspective, maintaining a
secure software environment is not just a necessity but a competitive
advantage. Companies that prioritize continuous security updates
demonstrate a commitment to data protection, thereby earning the
trust of their user base.
 • User security feedback integration: Continuous updates provide an
avenue to incorporate user feedback concerning security concerns.
This iterative process helps developers align their products with the
evolving security needs and preferences of their customers.

Meeting evolving user expectations

User expectations have evolved in tandem with technological advancements
and increased importance of cybersecurity. Today, users expect the
following from their digital experiences:
• Frequent enhancements: Users anticipate their software or apps to
continually improve. Whether it is a more intuitive interface, new
features, improved security measures, or enhanced performance,
they expect regular enhancements to the products they rely on.
• Timely bug fixes: Bugs and glitches are inevitable in any software,
but users expect swift resolutions. Continuous updates should
promptly address these issues to maintain user trust.
• Rapid vulnerability mitigation: Users demand that software
vulnerabilities be addressed promptly. Continuous updates ensure
that any newly discovered security vulnerabilities are swiftly
rectified to maintain user trust and data integrity.
• Proactive security updates: Users are increasingly aware of the
dangers posed by cybersecurity threats. They expect timely security
updates to protect their data.
• Transparency: Users appreciate clear communication regarding the
purpose and benefits of updates. Detailed release notes and
explanations about changes build trust and understanding.
Continuous updates are not a passing trend but an essential aspect of our
modern digital world.
They are pivotal for meeting user expectations in terms of ongoing
enhancements, security, and adaptability. Whether you are a software
developer, a business owner, or a consumer, recognizing the significance of
continuous updates is crucial. By embracing this concept and consistently
delivering on user expectations, we ensure that our digital experiences
remain relevant and valuable in a rapidly changing world. In today's digital
ecosystem, continuous updates are not optional; they are a necessity. They
serve as the linchpin for meeting user expectations regarding ongoing
improvements, protection against evolving cyber threats, and proactive
security measures. Also, acknowledging the crucial role of continuous
security updates is vital for software developers, business owners, and
consumers alike. Embracing this concept ensures that our digital
experiences remain secure, resilient, and trustworthy in a world marked by
relentless cybersecurity challenges.

Continuous uptime
In today's hyperconnected digital world, ensuring continuous uptime has
transitioned from a preference to an absolute necessity. Regardless of your
role, be it a business owner, a tech enthusiast, or a consumer, the quest for
uninterrupted digital services is no longer a luxury but a critical
requirement. Let us explore the significance of continuous uptime, backed
by compelling case studies and real-world examples, highlighting why it is
indispensable.

Understanding continuous uptime

Continuous uptime is a measure of the uninterrupted availability of digital
services, applications, and systems. It signifies the ability of these services
to remain operational and accessible to users without any significant
downtime. Unlike traditional models, which might accept planned
interruptions for updates and maintenance, continuous uptime aims to
provide services round the clock. To understand this in real-world scenario,
let us look at various case studies:
• Case study: Amazon Web Services
Amazon Web Services (AWS) is a prime example of a cloud
service provider that prioritizes continuous uptime. AWS promises
an impressive Service Level Agreement (SLA) with uptime
 percentages as high as 99.99%. This commitment ensures that
businesses relying on AWS can deliver their services without
disruption.
• Case study: Google's Search Engine
Google's search engine is renowned for its impressive uptime record.
Despite handling an astronomical volume of search queries every
day, Google's search engine experiences minimal, if any, downtime,
delivering search results without interruption to millions of users
worldwide.

Necessity of continuous uptime

Continuous uptime is not just a preference; it has become an imperative due
to several compelling reasons. Let us look at various reasons along with the
case studies for real-world:
• Meeting user expectations: In the digital age, users expect
uninterrupted access to their favorite services. Any downtime can
result in frustration and potentially drive users to competitors who
offer more reliable services.
Case study: Social media platforms
Social media platforms like Facebook, Twitter, and Instagram have a
global user base that expects real-time updates and engagement.
These platforms invest heavily in infrastructure to maintain
continuous uptime, ensuring users can access and interact with their
services at any time. In October 2021, Facebook and its subsidiary,
WhatsApp, experienced a widespread outage that lasted for several
hours. Millions of users around the world were unable to access
these platforms, which disrupted communication and access to
critical services. This outage exemplified the critical importance of
continuous uptime in meeting user expectations and avoiding user
frustration.
• Financial implications: Downtime can be financially catastrophic.
For e-commerce businesses, even a few minutes of downtime during
a peak shopping season can result in substantial revenue losses.
 Case study: E-commerce websites
During major shopping events like Black Friday, e-commerce
websites are under immense pressure to deliver a seamless shopping
experience. Continuous uptime is essential to prevent revenue losses
and maintain customer trust.
• Reputation and trust: The reputation of a business or service
provider is closely tied to its ability to maintain continuous uptime.
Users who experience frequent service interruptions are unlikely to
remain loyal and may voice their dissatisfaction online, further
damaging the brand's reputation.
Case study: Retail systems
In 2013, retail giant Target suffered a major data breach due to a
vulnerability in its payment card system. This breach led to a loss of
customer trust and tarnished Target's reputation. Continuous uptime
and robust security measures are essential to safeguard a company's
reputation and maintain customer trust.
• Operational efficiency: For businesses that rely on digital tools for
day-to-day operations, any interruption can disrupt workflows,
decrease productivity, and result in operational inefficiencies.
Case study: Financial institutions
Financial institutions, such as banks, depend on continuous uptime
to process transactions, provide access to online banking, and deliver
financial services. Even a brief interruption can have far-reaching
consequences, impacting customers and financial operations.
• Security and data integrity: In sectors like healthcare and finance,
continuous uptime is paramount to ensure that critical systems
remain accessible for timely and accurate care or financial
transactions. Downtime can lead to security vulnerabilities and data
loss.
Case study: Healthcare downtime
The healthcare sector relies on continuous uptime for electronic
health record systems, telemedicine platforms, and other critical
 systems. Downtime can lead to security vulnerabilities and data loss,
compromising patient care and data integrity.
• Global competitiveness: In a global market, businesses need to be
accessible to customers worldwide, across different time zones.
Continuous uptime is necessary to remain competitive and ensure a
global presence.
Case study: International banking services
International banks need to provide uninterrupted services to
customers in different time zones around the world. Continuous
uptime ensures global competitiveness and access to financial
services for customers, regardless of their location.
• Legal and regulatory compliance: Many industries are subject to
legal and regulatory requirements that demand data accessibility and
security. Continuous uptime is essential for compliance with these
regulations.
Case study: The General Data Protection Regulation
The General Data Protection Regulation (GDPR) in the European
Union mandates strict data access and security requirements.
Organizations must maintain continuous uptime to comply with
these regulations and avoid hefty fines for data breaches.
• Customer satisfaction and retention: Consistently providing
uninterrupted services enhances customer satisfaction and retention.
Customers are more likely to stay with a service provider they can
rely on.
Case study: Netflix
Streaming services like Netflix rely on continuous uptime to ensure
that customers can access content without interruptions. Consistently
providing uninterrupted services enhances customer satisfaction and
retention, as subscribers are more likely to stay with a service they
can rely on.
• Disaster recovery: In the event of a disaster or catastrophic event,
continuous uptime ensures that critical systems and data remain
 accessible, even when physical infrastructure is compromised.
Case study: Natural disasters
During natural disasters like Hurricane Katrina, continuous uptime is
essential for emergency services and critical systems to remain
operational. Downtime in such situations can have life-threatening
consequences.
Continuous uptime is not merely a technical requirement; it is a
fundamental business requirement in the digital age. In a world where
digital services have become integral to our lives, businesses and service
providers must prioritize continuous uptime to meet user expectations,
protect their brand reputation, safeguard financial stability, enable
operational efficiency and ensure data security and regulatory compliance.
Uptime is not a luxury; it is a commitment to excellence in a world where
digital services never sleep.

Continuous delivery and continuous deployments

We discussed in short about continuous delivery and continuous deployment
in Chapter 1, Lead the transformation with DevOps. In the realm of
software development, two closely related practices have gained significant
attention over the years: continuous delivery (CD) and continuous
deployment (CI/CD). These methodologies have revolutionized how
software is developed, tested, and released. While both share the objective
of delivering software faster and more efficiently, they operate with distinct
objectives, significance, and challenges. Let’s explore the differences
between continuous delivery and continuous deployment, discuss their
importance, shed light on the challenges that development teams face when
implementing these practices, and outline best practices for each.

Continuous delivery: Streamlining the delivery

pipeline
Continuous delivery, is a software development practice that aims to
streamline the delivery pipeline. Its primary focus is on automating the
build, test, and deployment processes. The core principle of CD is to ensure
that software is always in a deployable state, ready for release at any time.

Importance of continuous delivery

The importance of CD is as follows:
• Reduced risk: CD emphasizes regular integration, testing, and
deployment. This minimizes the chances of last-minute errors or
deployment failures, reducing risk.
• Faster time to market: CD accelerates the release cycle, enabling
companies to quickly push new features and updates to end-users,
gaining a competitive edge.
• Consistency: CD ensures that every build is consistently tested and
can be deployed without manual interventions, leading to a more
stable product.
• Improved collaboration: Teams that practice CD often collaborate
more effectively, breaking down silos between development and
operations.

Best practices for continuous delivery

The best practices for CD are listed as follows:
• Automation: Automate as many stages of the delivery pipeline as
possible, including testing, building, and deployment.
• Comprehensive testing: Implement robust automated testing,
including unit tests, integration tests, and user acceptance tests.
• Infrastructure as Code (IaC): Manage infrastructure and
configuration as code to ensure consistency across environments.
• Version control: Use version control systems like Git to track
changes and enable collaboration.
• Feature toggles: Implement feature toggles to control the release of
new features independently.
Challenges of continuous delivery
The following are some of the challenges of continuous delivery:
• Testing complexity: Implementing comprehensive automated
testing can be challenging, especially for complex software.
• Infrastructure management: Ensuring consistent infrastructure for
testing and deployment can be complex, particularly in
heterogeneous environments.
• Cultural shift: CD often requires a cultural shift in organizations,
encouraging more collaboration and automation.

Continuous deployment: Automating the release

pipeline
Continuous deployment, part of the broader CI/CD pipeline, takes
automation to the next level. In addition to the practices associated with
continuous delivery, continuous deployment aims to automatically release
software changes to production as soon as they pass automated tests.

Importance of continuous deployment

The following is the importance of continuous deployment:
• Rapid release: Continuous Deployment automates the process from
code commit to production, enabling rapid, continuous releases.
• Immediate user feedback: Users benefit from new features and
improvements almost immediately, and their feedback can drive
further enhancements.
• Efficiency: By automating the deployment process, CD eliminates
manual intervention, reducing the chances of human error and
saving time.
• Competitive advantage: Organizations practicing CD can swiftly
respond to market demands and stay ahead of the competition.

Best practices for continuous delivery

The best practices for continuous deployment are:
• Rigorous testing: Implement comprehensive automated testing,
including regression tests, to ensure code reliability.
• Gradual rollouts: Use feature flags and gradual rollouts to
minimize the impact of potential issues in production.
• Monitoring and alerting: Implement robust monitoring and alerting
systems to detect issues in real-time.
• Manual overrides: Include mechanisms for manual intervention in
case of emergencies.

Challenges of continuous delivery

The challenges of continuous deployment are:
• Risk management: With automated deployments, there is a higher
risk of introducing errors or bugs directly into production.
• Testing rigor: The testing process must be rigorous to ensure that
only reliable code reaches production.
• Compliance and security: In industries with strict compliance
requirements, Continuous Deployment must navigate regulatory
challenges.

Continuous delivery vs. continuous deployment

In Chapter 1, Chapter 1, Lead the transformation with DevOps we
discussed, in short, the key differences between continuous delivery and
continuous deployment (refer to Figure 1.8). Now, let us discuss how these
two approaches are different in the following aspects:
• Deployment to production:
○ Continuous delivery: In CD, the software is always in a
deployable state, but the decision to release it to production is
typically made manually. The release process can be initiated
when the team decides it is the right time.
 Continuous deployment: In CI/CD, the entire process from
○ code commit to production deployment is automated. Software
changes that pass automated tests are automatically deployed to
production, often multiple times a day.
• Human intervention:
○ Continuous delivery: CD often involves manual decision-
making to initiate the release to production. Teams have more
control and can choose when to release new features.
○ Continuous deployment: CI/CD minimizes human
intervention, allowing the software to be deployed to
production without manual approval once it passes automated
tests.
• Risk and speed:
○ Continuous delivery: CD strongly emphasizes reducing risk
through automation and testing while allowing teams to
maintain a degree of control over the release process. It is more
cautious.
○ Continuous deployment: CI/CD prioritizes speed and
automation. It is more aggressive in terms of releasing changes,
aiming for rapid delivery and immediate user feedback.
• Immediate user feedback:
○ Continuous delivery: CD does not guarantee immediate user
feedback. Users may experience new features or changes only
after the manual deployment to production.
○ Continuous deployment: CI/CD provides immediate user
feedback. Since deployments are automatic, changes are
quickly visible to users. This facilitates rapid iteration based on
user feedback.
• Level of control:
○ Continuous delivery: CD offers more control over the release
process. Deployment to production is a manual step, giving
 teams the ability to pause or adjust the release if necessary.
○ Continuous deployment: CI/CD reduces the level of control
over the release process. Once code passes automated tests, it is
automatically deployed, limiting the opportunity for manual
intervention.
Table 9.1 shows the comparison between continuous delivery and
continuous deployment, which will help you choose the right approach:

Continuous
Aspect Continuous delivery
deployment
Deployment to Manual decision to Fully automated
Production initiate release. deployment to
production.
Human Intervention Manual approval for Minimal to no
release. manual intervention
for release.
Risk and Speed Balanced approach, Emphasis on speed,
cautious release. rapid and automated
releases.
Immediate User Immediate user Immediate user
Feedback feedback not feedback due to rapid
guaranteed. releases.

Level of Control More control over Less control over the

the release process. release process.

Table 9.1: Continuous delivery vs continuous deployment

Choosing the right approach

The decision to adopt continuous delivery or continuous deployment
depends on a variety of factors, including the nature of the software, the
organization's culture, and its risk tolerance. Many organizations begin with
continuous delivery and, as their confidence and automation capabilities
grow, progress to continuous deployment. We can keep the following in
mind while choosing one or other:
• Continuous delivery: CD is suitable for organizations that value a
controlled release process, prioritize risk reduction, and want to
maintain a level of manual control over production deployments. It
is ideal when immediate user feedback is not the top priority.
• Continuous deployment: CI/CD is a better fit for organizations
aiming for rapid, automated releases, and immediate user feedback.
It is suitable when speed and efficiency in software delivery are
paramount, even if it means relinquishing some manual control.
Ultimately, whether you choose continuous delivery or continuous
deployment, both methodologies offer significant benefits, including faster
time-to-market, reduced risk, and improved collaboration. The adoption of
these practices is not only about adopting new tools and processes but also
embracing a cultural shift toward automation and collaboration. By
navigating the path to perfection, development teams can harness the power
of CD and CI/CD to deliver software more efficiently, with improved
quality and consistency, meeting the ever-evolving demands of the digital
landscape.

CI/CD pipeline example

Until now, in this chapter, we have discussed various aspects of continuous
delivery and continuous deployment. Now, take a look at the example of
continuous deployment to production. We will take the example from
Chapter 5, Continuous Integration and use the Jenkins file (Example 5.4 ) to
add the continuous deployment stage into it.
pipeline {
agent any
stages {
stage('Checkout') {
 steps {
git branch: 'main', url: '[Link]
}
}
stage('Build') {
steps {
sh 'mvn clean package'
}
}
stage('Test') {
steps {
sh 'mvn test'
}
post {
always {
junit 'target/Test-reports/**/*.xml'
}
}
}
stage('Deploy to Staging') {
steps {
sh [Link] #execute deployment script
}
}
 stage('User Acceptance Testing (UAT)') {
steps {
sh 'mvn uat-test' // Replace with your UAT testing command
}
post {
success {
// Notify on successful UAT
slackSend channel: '#notifications', color:'good', message: 'U
AT passed!'
}
failure {
// Notify on UAT failure
slackSend channel: '#notifications', color: 'danger', message:
'UAT failed!'
}
}
}
stage('Continuous Deployment to Production') {
when {
expression { [Link]('SUCCES
S') }
}
steps {
sh [Link] #execute deployment scr
ipt
 }
}
}

}
Example 9.1: CI/CD with Jenkins Pipeline (Jenkinsfile)

Here is what we have added to the existing pipeline:

• Deploy to staging: This stage deploys the application to a staging
server. Make sure to write a deployment script that matches your
deployment method.
• User Acceptance Testing (UAT): This stage runs UAT tests.
Replace the placeholder command with your actual UAT testing
command. It is common to notify the team of UAT results.
• Continuous deployment to production: This stage deploys the
application to the production environment automatically, without
manual intervention. It is conditioned on the success of previous
stages. The deployment to production occurs as soon as the code
passes the automated tests and UAT. Be cautious with this stage, as it
directly deploys to the production environment.

Conclusion
In this chapter, we explored the significance of continuous uptime, and
CI/CD in the software development world. Continuous delivery focuses on
automating the entire delivery pipeline, ensuring that the software is always
in a deployable state, ready for release at any time. Continuous deployment
takes automation to the next level, with an emphasis on fully automating the
deployment to production as soon as code passes automated tests. The
chapter outlined the key differences between these two approaches,
discussed their importance, and provided insights into best practices and
challenges associated with each. Additionally, a practical Jenkins pipeline
example was introduced, showcasing how to incorporate continuous
deployment into the development process, making software delivery more
efficient and reliable.
Since you now have a good understanding of application deployment, the
upcoming chapter discusses DevOps workflows specifically tailored for
mobile software development.

Join our book’s Discord space

Join the book’s Discord Workspace for Latest updates, Offers, Tech
happenings around the world, New Release and Sessions with the Authors:
[Link]
 CHAPTER 10
Mobile DevOps

Introduction
In the ever-evolving landscape of mobile app development, the need for
speed, quality, and efficiency is paramount. To meet these demands, Mobile
DevOps has emerged as a crucial methodology. It represents a pivotal shift
in the world of app development, optimizing the entire lifecycle from
concept to deployment. Mobile DevOps has become an indispensable
framework, shaping the way we create and deliver mobile applications. This
methodology not only accelerates the pace of app creation but also ensures a
higher standard of quality and user satisfaction.
By combining the fundamental principles of DevOps with a mobile-specific
focus, Mobile DevOps addresses the challenges posed by platform diversity,
user experience requirements, and the intricacies of app store deployment.
In this chapter, we will look at what Mobile DevOps is, how it differs from
traditional DevOps, and why Mobile DevOps is essential in today's fast-
paced mobile app development ecosystem. We will discuss DevOps
workflow for mobile and some best practices related to it.

Structure
In this chapter, we will discuss the following topics:
 • Introduction to Mobile DevOps
• DevOps vs. Mobile DevOps
• Necessity of Mobile DevOps
• DevOps workflow for mobile
• Mobile device fragmentation
• Challenges of implementing DevOps to mobile
• Mobile CI/CD pipeline in the cloud
• Mobile DevOps best practices
• Continuous testing on parallel devices

Objectives
By the end of this chapter, you will be equipped to implement Mobile
DevOps effectively and deliver high-quality mobile apps. You will gain a
clear understanding of Mobile DevOps and its pivotal role in mobile app
development, learning about how it differs from traditional DevOps and
why it holds such significance in mobile development. Additionally, you
will learn how to deal with challenges like device fragmentation, covering
OS variations, screen sizes, and Android's pixel density classifications.
Next, we will address hurdles in DevOps adoption for mobile and you will
gain knowledge on how to devise strategies to overcome them. A key focus
will be establishing a streamlined Mobile continuous integration and
continuous delivery (CI/CD) pipeline in the cloud for enhanced efficiency.
You will also become familiar with best practices in Mobile DevOps to
accelerate development and ensure top-notch quality. You will also gain
insights into the importance of continuous testing across various devices to
guarantee seamless performance.

Introduction to Mobile DevOps

In previous chapters, the dynamic realm of software development has been
discussed. DevOps which has established itself as a transformative
methodology, fostering collaboration between development and operations
teams and automating processes to accelerate delivery and improve quality.
When discussing DevOps comprehensively, we must include mobile
development and the realm of smartphones, an area witnessing the most
rapid growth in the domain of computer ownership. The last decade has
witnessed an astounding surge in smartphone usage, with billions of these
devices being possessed worldwide, as depicted in Figure 10.1. The
ownership of smartphones is projected to continue to grow, particularly in
developing nations. Presently, there are over 7.33 billion smartphone users
globally, and this number is anticipated to reach 7.49 billion by 2025,
establishing it as a market and user base of undeniable significance1. Refer
to the following figure:

Figure 10.1: Smartphone users globally

Moreover, smartphones possess a distinctive characteristic that renders

DevOps an indispensable practice: they belong to a category of internet-
connected devices where continual updates are not only welcomed but
anticipated by default. This is due to the fact that smartphones are primarily
aimed at non-technical consumers who seek a hassle-free experience with
minimal involvement in maintaining their devices. This phenomenon has
been significantly driven by the app ecosystem enveloping smartphones,
simplifying the process of downloading new software and receiving
updates, making it relatively low-risk for end users.

DevOps vs. Mobile DevOps

In our journey through DevOps principles, we have witnessed its
transformative power in streamlining software development and operations.
Now, let us explore how Mobile DevOps is different from traditional
DevOps and how Mobile DevOps takes a step further, catering to the unique
demands of mobile app development in the age of smartphones and tablets.
DevOps is a well-established approach that unites software development
(Dev) and IT operations (Ops) teams, breaking down silos to enhance
collaboration and automate key processes. The overarching goal of DevOps
is to deliver software more rapidly, reliably, and efficiently. Key principles
of DevOps include continuous integration, continuous delivery, automated
testing, and infrastructure as code. We have already gone through all these
key terminologies in previous chapters.
Mobile DevOps, on the other hand, recognizes that mobile app development
has unique characteristics that demand a specialized approach. It is a
precision-tailored approach to address the distinctive intricacies of mobile
app development. In the age of smartphones and tablets, where users
demand seamless, high-performance mobile experiences, Mobile DevOps
has emerged as a critical framework.
It takes the well-established DevOps principles and customizes them to
meet the unique challenges of mobile development. By prioritizing platform
diversity, user experience, app store deployments, and real device testing,
Mobile DevOps ensures that mobile applications are not just created but
perfected to meet the ever-evolving expectations of the mobile-savvy user.
Mobile DevOps is the key to delivering excellence in every app in a world
driven by mobile technology.
Here is how it differs from traditional DevOps:
 • Platform diversity: Mobile app development involves two major
platforms: iOS and Android. Each has its own programming
languages, development tools, and release processes. Mobile
DevOps takes this into account and adapts DevOps practices for
each platform.
• User Experience: Mobile apps are highly focused on user
experience. Mobile DevOps places a strong emphasis on User
Experience (UX) testing, ensuring that the app's performance and
user interface meet high standards.
• App store deployment: Deploying mobile apps to app stores (e.g.,
Apple's App Store and Google Play Store) comes with unique
challenges and requirements. Mobile DevOps includes automated
processes for app submission, review, and updates.
• Real devices and simulators: Mobile app testing often involves real
devices as well as simulators or emulators. Mobile DevOps
integrates these aspects to facilitate comprehensive testing.

Necessity of Mobile DevOps

Mobile DevOps is vital for several reasons:
• Accelerated development: Mobile DevOps speeds up app
development, reducing time-to-market for new features and updates.
• Enhanced quality: Automated testing ensures consistent app
functionality and performance, reducing the likelihood of releasing
flawed software.
• Seamless collaboration: Collaboration between development and
operations teams ensures efficient app updates, bug fixes, and
feature enhancements.
• User-centric approach: Mobile DevOps puts the user at the center
of development, enabling user feedback to drive app improvements.
• Security and compliance: Mobile apps often handle sensitive user
data, and Mobile DevOps includes automated security checks to
 identify and mitigate vulnerabilities while ensuring compliance with
data protection regulations.

DevOps workflow for mobile

The DevOps workflow for mobile app development represents a strategic
approach of DevOps in the mobile application development process. This
methodology optimizes the entire app development lifecycle, from planning
and coding to testing, deployment, and continuous monitoring and
improvement. By implementing DevOps in mobile development, teams can
ensure faster delivery, higher app quality, and a more responsive approach to
user feedback and evolving mobile technologies. It emphasizes automation,
collaboration, and a user-centric perspective, all of which are critical in
meeting the ever-increasing demands of the fast-paced and dynamic world
of mobile app development.
The fusion of DevOps practices with mobile app development has
revolutionized the way we create, test, and deploy applications. Let us
explore the intricacies of the DevOps workflow for mobile, focusing on the
stages of the DevOps pipeline for Android devices. This methodology
allows teams to streamline development processes and deliver high-quality
mobile apps efficiently. Here are some common stages in the DevOps
workflow for mobile:
• Plan: The journey begins with a well-defined plan. This stage
involves identifying the goals and requirements of the Android
mobile app, understanding user needs, and creating a roadmap for
development. Key tasks include defining the scope, creating user
stories, and setting development milestones.
• Code: Once the plan is in place, the coding phase commences.
Android app developers use Java, Kotlin, or other programming
languages to write the app's code. During this stage, collaborative
version control systems like Git are essential for managing code
changes and ensuring that the entire development team is on the
same page.
 Build: The build stage involves compiling the source code into
• executable files for Android devices. Build automation tools like
Gradle simplify this process, ensuring that the code is ready for
testing and eventual deployment.
• Test: Testing is a critical phase in the Android DevOps pipeline. It
includes unit testing, integration testing, and UI testing on real
devices and emulators. Automated testing frameworks like Espresso
and Appium are commonly used to assess the app's functionality,
performance, and compatibility across various Android versions and
devices.
• Deploy: Deploying an Android app to the Google Play Store
requires a well-structured process. In this stage, DevOps practices
come into play as automated deployment scripts are used to package
the app, handle code signing, and release it to the Google Play Store.
Continuous integration and continuous delivery (CI/CD) tools
ensure that the deployment process is both rapid and reliable.
• Monitor and optimize: After deployment, the DevOps pipeline
does not end; it transitions into a continuous monitoring and
optimization phase. This step is vital for ensuring that the app
remains stable and performs well in the real world. Tools like
Google Play Console and crash reporting services provide insights
into app performance, user feedback, and crash reports. DevOps
teams use this data to identify and rectify issues promptly. In essence
apps cannot be directly monitored in production like traditional
DevOps monitoring because apps run on the user’s device. So,
Mobile DevOps focuses on pre-deployment testing and post-
deployment monitoring with analytics using tools like Google Play
console and Firebase Crashlytics.

Mobile device fragmentation

While developing mobile apps, the challenge of fragmentation looms large,
particularly in the dynamic landscapes of iOS and Android. The controlled
ecosystem of iOS, overseen by Apple, contrasts sharply with open and
diverse terrain of Android. Let us explore the intricacies, challenges, and
best practices associated with iOS and Android fragmentation, building for
disparate screens, and the nuances of hardware and 3D support.

iOS vs. Android fragmentation

In iOS, Apple's meticulous control results in a limited number of device
models and a streamlined set of hardware features. Since the debut of the
first iPhone in 2007, only 42 different devices have emerged. This control
extends to operating system updates, ensuring swift adoption of new
versions across supported devices. In contrast, Android's open ecosystem
welcomes a multitude of manufacturers customizing everything from screen
size to hardware sensors. With over 24,000 devices from 1,300
manufacturers, Android presents a fragmentation challenge of a magnitude
1,000 times greater than iOS, complicating testing endeavors significantly.

OS fragmentation for iOS

In mobile app development, iOS stands out for its curated ecosystem,
orchestrated by Apple's meticulous control. However, even within this
controlled environment, the concept of OS fragmentation persists, although
in a more nuanced manner than its Android counterpart. Let us explore the
dynamics of iOS OS fragmentation and how developers can navigate this
mosaic for a seamless user experience.

The orchestrated iOS ecosystem

Apple's approach to iOS development is akin to a carefully conducted
symphony. The company maintains strict control over both hardware and
software, ensuring a harmonious experience across its devices. Unlike
Android, where a multitude of manufacturers contribute to a vast array of
devices, iOS devices are limited to a curated selection produced by Apple.
This controlled environment theoretically minimizes fragmentation,
providing developers with a consistent platform to target. However, the iOS
landscape is not devoid of variations. The intricacies of OS versions, device
models, and user adoption rates contribute to a subtle form of fragmentation
that demands developers' attention. While Apple ensures swift adoption of
major OS updates across supported devices, the nuances of legacy device
usage and user preferences can introduce challenges.

iOS version adoption

One of the defining features of the iOS ecosystem is the rapid adoption of
new OS versions. Apple's ability to push updates simultaneously to a large
percentage of supported devices ensures that users swiftly upgrade to the
latest features and security patches. For example, iOS 14 achieved an
impressive 86% adoption rate within seven months of its release, iOS 16
achieved 68.95% in three months, and iOS 17 achieved a 23% adoption rate
within 18 days of its release. All these showcase the efficiency of Apple's
update distribution.
This swift adoption rate is a boon for developers aiming to leverage the
latest iOS features, but it also sets a high bar for ensuring backward
compatibility. While the majority of users embrace the latest iOS versions, a
fraction may linger on older versions due to device limitations or personal
preferences. Developers must carefully balance innovation with
compatibility, catering to both early adopters and those on older devices.

Legacy devices and tailored experiences

The iOS ecosystem, while streamlined, is not immune to the challenges
posed by legacy devices. As Apple introduces cutting-edge features in
newer OS versions, ensuring a seamless experience across devices with
varying capabilities becomes crucial. For instance, a developer designing an
app for iOS 16 must consider how it will perform on an iPhone 6 running
iOS 13, balancing modern features with performance on older hardware.
The challenge intensifies when considering devices that are no longer
eligible for the latest OS updates. Users holding onto older iPhones, such as
the iPhone 5s or earlier models, may be restricted to outdated iOS versions.
Crafting applications that cater to this diverse user base, with a spectrum of
OS versions and device capabilities, demands a strategic approach from
developers.
Best practices for iOS fragmentation
In the symphony of iOS development, understanding and addressing OS
fragmentation is the key to delivering a harmonious user experience. While
Apple's controlled ecosystem provides a solid foundation, the variations in
OS versions and device capabilities necessitate a nuanced approach from
developers. By adopting best practices, embracing progressive
enhancement, and testing rigorously across the iOS spectrum, developers
can navigate the mosaic of iOS OS fragmentation and ensure their apps
resonate seamlessly across the diverse iOS user base. Here are a few best
practices while navigating iOS fragmentation:
• Version-agnostic design: Adopting a version-agnostic design
approach ensures that the app's core functionalities remain consistent
across various iOS versions. While leveraging new features is
enticing, ensuring a seamless experience for users on older OS
versions is paramount.
• Progressive enhancement: Embrace progressive enhancement to
gracefully incorporate advanced features for users on the latest iOS
versions while providing essential functionality for those on older
versions. This approach caters to a broad audience without
compromising on innovation.
• Thorough testing on legacy devices: Testing on a range of devices,
including legacy models with older iOS versions, is crucial. This
ensures that the app performs optimally across the spectrum of
devices that constitute the iOS user base.
• User education and communication: Proactively communicate
with users about the importance of updating to the latest iOS version
for an optimal experience. User education can contribute to a more
homogeneous user base and simplify the development landscape.

Navigating the complexity of OS fragmentation

for Android
Android, the dynamic and open-source mobile operating system, boasts a
vast ecosystem that thrives on diversity. However, this diversity introduces a
unique challenge for developers: Android OS fragmentation. Unlike its iOS
counterpart, Android spans a multitude of devices from various
manufacturers, resulting in a complex mosaic of OS versions, screen sizes,
and hardware configurations.
Let us look into the intricacies of Android OS fragmentation and explore
strategies for developers to master this diverse landscape:
• The symphony of Android versions: Android version
fragmentation operates on two levels: major Android versions and
OEM-driven customizations. The Android market's complexity
arises from the diverse release cycles of major Android updates and
the additional layer of customization introduced by original
equipment manufacturers (OEMs).
Unlike Apple's centralized control, where iOS updates are
universally pushed to supported devices, Android updates face a
more intricate journey. Major vendors may provide only a limited
number of OS version updates for their devices, and smaller
manufacturers might not offer updates at all. The result is a diverse
user base running a spectrum of Android versions, presenting
challenges for developers aiming to deliver a consistent experience.
• Navigating the Android OS Symphony: To understand the
magnitude of the Android OS fragmentation challenge, consider the
distribution of users across different Android versions. Google's
data, as of September 2023, revealed a scattered landscape with a
significant portion of users still operating on Android 4.x releases,
which are more than nine years old. Achieving widespread adoption
of the latest Android versions becomes a strategic dance for
developers, requiring them to balance innovation with backward
compatibility.
For instance, to reach an adoption level comparable to the latest iOS
version, developers need to support at least Android 5.1 Lollipop,
released in 2014. This dynamic highlight the intricate dance
 developers must perform to cater to a diverse user base while
incorporating the latest Android features.
• Android customizations with diversity: The Android OS's open
nature allows OEMs to customize the user experience on their
devices. This customization introduces variations in the user
interface, pre-installed apps, and additional features. While this
diversity allows manufacturers to differentiate their products, it adds
a layer of complexity for developers.
Testing an application on one device per major Android version is
not sufficient due to the variations introduced by OEM
customizations. Each device becomes a unique entity with its own
set of performance, security, and potential bug variations, requiring
developers to navigate a complex landscape to ensure a seamless
user experience.
Google addressed this with Android 8.0 Oreo, introducing a new
hardware abstraction layer. This layer allows device-specific code to
run outside the kernel, reducing the redevelopment and testing
required for OS upgrades. However, widespread adoption of these
improvements remains dependent on OEMs, contributing to the
persistence of fragmentation challenges.

Android for disparate screens

In the realm of diverse hardware manufacturers boasting over 24,000
models, screen sizes and resolutions exhibit significant variations. A prime
example is the HP Slate 21, flaunting a massive 21.5-inch touchscreen, and
the Samsung Galaxy Z Fold, featuring a 2316 × 904 cover display that
unfolds to unveil a 2176 × 1812 double-wide inner display.
Beyond the spectrum of screen sizes, an ongoing competition centers around
achieving higher pixel density. This pursuit results in clearer text and
sharper graphics, ultimately enhancing the viewing experience. The Sony
Xperia Z5 currently leads in pixel density, sporting a 3840 × 2160 UHS-1
display within a compact 5.2-inch screen, yielding an impressive 806 pixels
per inch (PPI). This approaches the upper limit of human eye resolution
discernment.
Applied Materials, a prominent producer of LCD and OLED displays,
conducted research on human perception of pixel density on handheld
devices. Findings indicate that at a distance of 4 inches from the eye, an
individual with 20/20 vision can distinguish up to 876 PPI. Consequently,
smartphone displays are nearing the theoretical limit of pixel density, though
emerging form factors like virtual reality headsets may propel density
advancements further.
The diversity in Android devices extends beyond OS versions to screen
sizes and resolutions, creating an additional layer of complexity for
developers. Android categorizes screens into different pixel density ranges,
from ldpi to xxxhdpi, each requiring a meticulous approach to ensure a
visually harmonious experience. To handle variation in pixel densities,
Android categorizes screens into the following pixel density ranges:
• ldpi (~120 dpi): Used on very low-resolution devices, such as the
HTC Tattoo and Motorola Flipout, with a screen resolution of 240 ×
320 pixels.
• mdpi (~160 dpi): The original screen resolution for Android
devices, seen in devices like the HTC Hero and Motorola Droid.
• tvdpi (~213 dpi): A resolution intended for televisions, like the
Google Nexus 7, but not considered a primary density group.
• hdpi (~240 dpi): Found in the second generation of phones, such as
the HTC Nexus One and Samsung Galaxy Ace, with a 50% increase
in resolution.
• xhdpi (~320 dpi): Adopted by phones like the Sony Xperia S,
Samsung Galaxy S III, and HTC One, featuring a 2x resolution.
• xxhdpi (~480 dpi): Initially seen in the Nexus 10 by Google, this
category represents devices with 3x scale.
• xxxhdpi (~640 dpi): The highest resolution used by devices like the
Nexus 6 and Samsung Galaxy S6 Edge, featuring a 4x scale.
Crafting applications that seamlessly scale across this spectrum demands
adherence to best practices such as:
• Density-independent and scalable pixels: Leveraging density-
independent pixels (dp) and scalable pixels (SP) ensures that UI
elements adapt gracefully to varying screen densities, providing a
consistent user interface.
○ Density-independent pixels (dp): Pixel unit that adjusts based
on the resolution of the device. For an mdpi screen, 1 pixel (px)
= 1 dp. For other screen resolutions, px = dp × (dpi / 160).
○ Scalable pixels (SP): Scalable pixel unit used for text or other
user-resizable elements. This starts at 1 sp = 1 dp and adjusts
based on the user-defined text zoom value.
• Provide alternate bitmaps for all available resolutions: Android
facilitates the use of alternate bitmaps tailored for various
resolutions. Achieve this by placing them in subfolders labelled
drawable-?dpi, where ?dpi represents the supported density ranges (
i.e idpi,hdpi). Similarly, for your app icon, employ subfolders named
mipmap-?dpi. This ensures that resources are retained during the
build of density-specific APKs, crucial as app icons are frequently
upscaled beyond the device resolution.
• Comprehensive testing: Given the wide variety of screen
resolutions, comprehensive testing on devices with differing
resolutions becomes imperative. Google's data on Android screen
size and density distribution aids developers in focusing their testing
efforts on prevalent resolutions.
• Use vector graphics whenever possible: In Android Studio, there is
a handy tool called Vector Asset Studio. It enables you to transform
Scalable Vector Graphics (SVG) or Photoshop Document (PSD)
files into Android Vector files, which can be utilized as resources in
your application. This ensures a visually appealing experience across
the diverse range of Android screens. Building applications that
seamlessly adapt to different screen sizes and resolutions requires
meticulous testing across devices with different resolutions. To
 streamline your testing process, Google offers user-mined data on
the usage of various device resolutions, as illustrated in Table 10.1:
xxhd
ldpi mdpi tvdpi hdpi xhdpi Total
pi
Small 0.40 0.40
% %
Norm 0.10 0.30 7.90 45.40 23.20 76.90
al % % % % % %
Large 1.20 3.40 1.10 6.80 1.70 14.20
% % % % % %
Xlarg 4.30 0.10 3.80 0.30 8.50
e % % % % %
Total 0.00 5.60 3.80 12.80 52.90 24.90
% % % % % %
Table 10.1: Android screen size and density distribution

Upon closer inspection, it becomes evident that certain resolutions are not
widely adopted. Unless your application explicitly caters to these users or
legacy devices, consider removing them from your device-testing matrix.
Notably, the ldpi density, with less than 0.1% market share, making it less
critical for optimization. Similarly, the tvdpi resolution represents a niche
market with just 3.8% usage. Since Android automatically downscales hdpi
assets to fit this screen resolution, it can be safely disregarded.
This refined approach narrows the focus to five essential device densities,
though it opens the door to an array of screen resolutions and aspect ratios
for testing. Strategies for testing will be explored later, but a combination of
emulated and physical devices is likely the key to ensuring an optimal user
experience across the diverse Android ecosystem.

Advanced hardware and 3D support

The inaugural Android device, the HTC Dream (also known as T-Mobile
G1), featured a 320 × 480 px medium-density touchscreen, a hardware
keyboard, speaker, microphone, five buttons, a clickable trackball, and a
rear-mounted camera. Although considered primitive by contemporary
standards, it served as an excellent launchpad for Android, particularly
when software keyboards were not yet supported.
In stark contrast, the modern Samsung Galaxy S21 Ultra 5G showcases a
3200 × 1440 resolution screen and a robust array of sensors, including a 2.9
GHz 8-core processor, Arm Mali-G78 MP14 GPU supporting Vulkan 1.1,
OpenGL ES 3.2, and OpenCL 2.0, five cameras, three microphones, stereo
speakers, an ultrasonic fingerprint reader, and various other sensors.
Flagship Samsung phones, positioned at the high end of the spectrum
encompass nearly all supported sensor types. Conversely, mass-market
phones might opt for less powerful chipsets and exclude certain sensors to
manage costs. Android utilizes data from physical sensors to generate
virtual sensors in software, such as the game rotation vector, gravity,
geomagnetic rotational vector, linear acceleration, rotation vector,
significant motion, and step detector/counter. The availability of these
virtual sensors depends on the presence of a sufficient set of physical
sensors, with variations in precision based on sensor inclusion.
While hardware sensors can be emulated, simulating real-world conditions
for testing proves challenging. The extensive diversity in hardware chipsets
and System on Chip (SoC) vendor-driver implementations necessitates a
comprehensive testing matrix to verify applications across a range of
devices.
Another crucial aspect for developers, especially in gaming, is 3D API
support. The evolution from OpenGL ES 1.1 to modern versions like
OpenGL ES 3.2 has been rapid. OpenGL ES 2.0 introduced a shift to a
programmable pipeline, enabling more direct control through shaders.
Subsequent versions improved performance and hardware independence,
supporting features like vertex array objects and instanced rendering.
The adoption of OpenGL ES has been widespread, with the majority of
modern devices (80.24%) supporting OpenGL ES 3.2 (Figure 10.2).
Meanwhile, Vulkan, a newer graphics API, has gained traction, providing
portability between desktop and mobile devices. Though not as swiftly
adopted as OpenGL ES, around 80% of Android devices now have some
level of Vulkan support. Refer to Figure 10.3:

Figure 10.2: OpenGL ES adoption2

Figure 10.3: Vulkan adoption3

Given the diverse implementation of 3D chipsets by various manufacturers,

thorough device testing on different phone models becomes imperative for
identifying bugs and performance issues, we will elaborate this in the
coming sections.

Challenges of implementing DevOps to mobile

DevOps, a methodology that emphasizes collaboration and communication
between software development and IT operations, has proven to be
immensely beneficial in the realm of traditional software development.
However, when it comes to mobile app development, unique challenges
arise that demand a tailored approach. Implementing DevOps in the mobile
domain is not without hurdles, and understanding these challenges is crucial
for a successful deployment. Let us look at them:
• Diverse ecosystems: One of the primary challenges in mobile
DevOps stems from the diverse ecosystems of iOS and Android. iOS
devices are tightly controlled by Apple, with a limited range of
models and screen sizes. On the contrary, Android supports various
devices from various manufacturers, each with distinct screen sizes,
resolutions, and hardware configurations. DevOps practices need to
accommodate this diversity to ensure consistent app performance
across a broad spectrum of devices.
• Frequent platform updates: Mobile platforms frequently release
updates, introducing new features, security patches, and changes to
the underlying infrastructure. Coordinating the release of mobile
apps with these platform updates poses a challenge. Mobile DevOps
teams must stay abreast of the latest platform changes and adapt
their workflows to ensure seamless integration with new operating
system versions while maintaining backward compatibility.
• Continuous testing for fragmentation: Fragmentation is a
significant issue in the Android ecosystem, with many devices
running different Android versions and customized user interfaces.
Testing applications across this fragmentation landscape is complex,
and Mobile DevOps teams must invest in robust testing strategies to
ensure the app functions seamlessly across diverse devices and OS
versions.
 User experience concerns: Mobile apps are highly user-centric, and
•
any disruption in the user experience can lead to negative reviews
and uninstallations. DevOps practices need to prioritize user
experience testing across various devices, screen sizes, and network
conditions. Ensuring the app performs well in real-world scenarios is
critical for user satisfaction.
• Limited resources on mobile devices: Mobile devices, compared to
traditional servers, have limited resources, including processing
power, memory, and storage. Optimizing applications for resource
efficiency is a crucial consideration in Mobile DevOps. Teams need
to adopt strategies such as code splitting, resource caching, and
efficient memory management to deliver responsive and resource-
efficient mobile applications.
• Security and compliance: Mobile apps often handle sensitive user
data, making security and compliance paramount. Implementing
DevOps practices that address security concerns throughout the
development lifecycle is essential. This includes secure coding
practices, regular security testing, and compliance with privacy
regulations, which vary across regions.
• App store approval processes: The Apple App store, and Google
Play store have rigorous approval processes to maintain quality and
security. DevOps teams must align their release cycles with these
approval processes, which can introduce delays. Coordination and
planning are essential to ensure a smooth transition from
development to deployment.
• Overcoming network variability: Mobile devices operate in
diverse network conditions, including 3G, 4G, and various levels of
Wi-Fi connectivity. DevOps practices must account for network
variability to ensure that mobile apps perform consistently across
different network environments. Implementing effective strategies
for offline functionality and optimizing data usage become crucial in
this context.
While the principles of DevOps remain valuable in mobile app
development, the unique challenges of the mobile landscape demand a
nuanced and adaptable approach. Overcoming these challenges requires
collaboration, continuous learning, and a commitment to enhancing the
Mobile DevOps pipeline to meet the evolving demands of the mobile
ecosystem.

Mobile CI/CD pipeline in the cloud

CI/CD is integral to the rapid and reliable deployment of mobile
applications. In the contemporary landscape, leveraging the cloud for
building and automating mobile CI/CD pipelines has become a game-
changer. Let us explore the components and steps involved in setting up a
mobile CI/CD pipeline in the cloud, accompanied by a real-world example.
Let us look at the components of a Mobile CI/CD pipeline:
• Source code management:
○ Example: Git
○ Role: Manages the source code repository and tracks changes.
• Build automation:
○ Example: Jenkins, Travis CI
○ Role: Automates the compilation of source code into
executable binaries.
• Automated testing:
○ Example: Appium, XCTest, Espresso
○ Role: Conducts automated testing to ensure code quality and
prevent regressions.
• Artifact repository:
○ Example: Artifactory, Nexus
○ Role: Stores and manages build artifacts, such as APKs and
IPA files.
 • Configuration management:
○ Example: Ansible, Puppet
○ Role: Manages configuration settings for different
environments.
• Deployment automation:
○ Example: Fastlane
○ Role: Automates the deployment of mobile apps to various
platforms.
• Continuous monitoring:
○ Example: Firebase Performance Monitoring, New Relic
○ Role: Monitors app performance and user experience in real-
time.
Now, let us look into the steps to set up a mobile CI/CD pipeline in the
cloud:
1. Choose a cloud provider:
a. Example: Amazon Web Services (AWS), Google Cloud
Platform (GCP)
b. Role: Provides infrastructure and services to host and run your
CI/CD pipeline.
2. Create a build environment:
a. Example: AWS CodeBuild, Google Cloud Build
b. Role: Configures a scalable environment to build your mobile
app.
3. Integrate with version control:
a. Example: Connect Git repository to the CI/CD platform.
b. Role: Triggers the pipeline on code changes.
4. Automate testing:
 a. Example: Use Appium for cross-platform mobile app testing.
b. Role: Ensures the application functions as expected and
validates new changes.
5. Build artifacts and store:
a. Example: Use Artifactory to store APKs and IPA files.
b. Role: Manages artifacts for easy retrieval and distribution.
6. Configure environments:
a. Example: Use Ansible for environment configuration.
b. Role: Ensures consistent configurations across different stages.
7. Deploy to app stores or devices:
a. Example: Utilize Fastlane to automate app deployment.
b. Role: Streamlines the process of releasing the app to app stores
or devices.
8. Monitor performance:
a. Example: Integrate Firebase performance monitoring.
b. Role: Monitors app performance in real-world scenarios.

Real-world example
Consider a scenario where a mobile development team uses AWS as their
cloud provider. They configure an AWS CodeBuild project to build their
React Native mobile app. Upon each commit to the Git repository,
CodeBuild triggers the build process, compiles the source code, runs
automated tests using Appium, and produces the APK.
The artifacts are then stored in an Artifactory repository. Ansible is
employed to manage different configurations for development, staging, and
production environments. Fastlane scripts automate the deployment process,
pushing the app to the Google Play Store for Android devices.
Firebase performance monitoring is integrated to track the app's
responsiveness and user experience. Any issues detected in the monitoring
phase can trigger automatic rollback or alert the development team.
This real-world example demonstrates the orchestration of various tools and
cloud services to create a seamless mobile CI/CD pipeline. Such a pipeline
enhances efficiency, accelerates release cycles, and ensures the delivery of
high-quality mobile applications to end-users.
Beloa is a Jenkins pipeline (Example 10.1) script sample that includes
various stages. This assumes you have the necessary Jenkins plugins
installed, especially those related to AWS and Docker:
pipeline {
agent any
environment {
AWS_REGION = 'your-aws-region'
S3_BUCKET = 'your-artifact-bucket'
}
stages {
stage('Checkout') {
steps {
checkout scm
}
}
stage('Build') {
steps {
script {
sh 'npm install'
 sh 'npm run build'
}
}
}
stage('Upload to S3') {
steps {
script {
withAWS(region: env.AWS_REGION, credentials: 'your-
aws-credentials') {
sh "aws s3 cp ./build/[Link] s3://${env.S3_BUCKE
T}/[Link]"
}
}
}
}
stage('Deploy to App Store') {
steps {
script {
// Add deployment steps here using any deployment tools
}
}
}
stage('Monitor Performance') {
steps {
 script {
// Integrate with any monitoring tools
}
}
}
}
post {
always {
// Clean up or perform any post-build actions
}
}
}
Example 10.1. Jenkins pipeline

Make sure to replace placeholder values such as your-aws-region, your-

artifact-bucket, and your-aws-credentials with your actual AWS region, S3
bucket name, and AWS credentials ID in Jenkins. This pipeline assumes you
have configured AWS credentials in Jenkins and have the necessary
permissions to interact with AWS services. Additionally, you would need to
integrate specific deployment steps in the Deploy to App Store stage based
on the tools you use for deployment.

Mobile DevOps best practices

Mobile DevOps, the marriage of development and operations for mobile
application delivery, has become indispensable in the ever-evolving
landscape of mobile app development. To ensure the seamless integration of
code changes, faster releases, and optimal user experiences, adopting
Mobile DevOps best practices is paramount. Let us look at them in detail:
• Early and frequent testing:
 ○ Importance: Detecting issues early in the development cycle
minimizes the chances of defects reaching the production stage.
○ Best practice: Implement automated testing throughout the
development process, covering unit tests, integration tests, and
UI tests.
• CI/CD pipeline:
○ Importance: Automating the build and deployment processes
ensures swift and consistent delivery of new features and bug
fixes.
○ Best practice: Employ CI/CD pipelines to automate code
integration, testing, and deployment, reducing manual errors
and enhancing efficiency.
• Version control:
○ Importance: Efficient version control is the backbone of
collaboration and allows for seamless rollbacks in case of
issues.
○ Best practice: utilize version control systems (e.g., Git) to
manage code versions, and establish branching strategies for
organized development.
• Environment standardization:
○ Importance: Consistent environments for development,
testing, and production prevent discrepancies and enhance
reliability.
○ Best practice: Use infrastructure-as-code (IaC) principles to
standardize and automate the creation of development and
testing environments.
• Monitoring and analytics:
○ Importance: Real-time monitoring provides insights into app
performance, user behavior, and potential issues.
 ○ Best practice: Integrate analytics tools and monitoring
solutions to track key metrics, identify performance
bottlenecks, and gather user feedback.
• Security integration:
○ Importance: Security is a non-negotiable aspect of mobile app
development to safeguard user data and ensure compliance.
○ Best practice: Integrate security checks into the CI/CD
pipeline, conduct regular security audits, and stay updated on
security best practices.
• Collaboration and communication:
○ Importance: Seamless communication and collaboration
among development, operations, and other stakeholders
streamline the development lifecycle.
○ Best practice: Utilize collaboration tools, establish clear
communication channels, and foster a culture of cross-
functional collaboration.
• Scalability planning:
○ Importance: As the user bases grow, scalability becomes
critical to handle increased traffic and maintain app
performance.
○ Best practice: Design architecture with scalability in mind,
regularly assess performance, and conduct load testing to
identify and address potential scalability challenges.
• Feedback loops:
○ Importance: Continuous feedback loops facilitate
improvement by incorporating user feedback and monitoring
app performance.
○ Best practice: Implement mechanisms for gathering user
feedback, conduct retrospective meetings, and leverage
feedback to drive continuous improvement.
 • Documentation and knowledge sharing:
○ Importance: Comprehensive documentation and knowledge
sharing ensure that the development team is aligned and can
address issues efficiently.
○ Best practice: Document code, processes, and configurations
and encourage knowledge sharing through documentation, team
training, and regular knowledge-sharing sessions.
Adopting these Mobile DevOps best practices fosters a culture of
collaboration, innovation, and efficiency in mobile app development. By
integrating these practices into the development lifecycle, teams can not
only deliver high-quality apps but also respond quickly to market demands
and provide exceptional user experiences. Mobile DevOps is not just a
methodology; it is a commitment to excellence in every phase of the mobile
app journey.

Continuous testing on parallel devices

In the dynamic realm of mobile app development, diversity reigns supreme
with a myriad of devices, operating systems, and screen sizes. The
traditional manual testing approach becomes a daunting task, consuming
time and prone to errors, especially when coupled with the relentless
demand for rapid releases and updates. Success in mobile app development
necessitates more than just innovation and user-centric design; it requires a
robust testing strategy capable of ensuring a flawless user experience across
an extensive array of devices.
A transformative solution emerges in the form of continuous testing on
parallel devices, a linchpin strategy for agile mobile app development teams
striving for high-quality releases. By harnessing the power of parallel
execution, cloud-based testing, and emulator farms, testing processes are
streamlined, time-to-market is accelerated, and the quality of mobile
applications is elevated.
Continuous testing on different devices is more than just testing; it is a
commitment to make sure your mobile app works well for everyone, no
matter their device. This method not only makes testing faster but also
guarantees that the mobile app stays strong and robust on all kinds of
devices.
Continuous testing involves integrating testing early and often in the
development lifecycle. When applied to mobile app development, it
becomes imperative to test on multiple devices concurrently to match the
diversity of the user base. Here are some ways of performing testing in
mobile app development:
• Parallel execution:
○ Example: Appium, Selenium Grid for mobile testing
○ Explanation: Parallel execution allows running tests
simultaneously on multiple devices. For instance, a test suite
can be divided, and each segment can run concurrently on
different devices. This significantly reduces testing time.
• Cloud-based testing services:
○ Example: Sauce Labs, BrowserStack, AWS Device Farm
○ Explanation: Leveraging cloud-based testing services enables
teams to run tests on a multitude of real devices or emulators in
parallel. This flexibility ensures coverage across various
devices and operating system versions.
• Emulator farms:
○ Example: Genymotion Cloud, Firebase Test Lab
○ Explanation: Emulator farms provide scalable and parallelized
testing environments. These platforms allow running tests on
emulated devices, ensuring coverage for a wide array of
scenarios without the need for physical devices.
Now, let us look at a real-world example using the above methods of
continuous testing. Consider a mobile development team working on a
fitness-tracking app compatible with both Android and iOS devices. With
continuous testing on parallel devices, they integrate the following
practices:
• Parallel execution using Appium:
○ The team divides their test suite into multiple threads, each
targeting a specific device and operating system combination.
○ Tests for various functionalities such as tracking workouts,
syncing data, and push notifications run concurrently on
emulators and real devices.
• Cloud-based testing with BrowserStack:
○ The team utilizes BrowserStack to run tests on real devices
hosted in the cloud.
○ Tests cover different devices and OS versions to ensure
compatibility with the diverse user base.
• Emulator farms for comprehensive coverage:
○ They integrate Genymotion Cloud into their CI/CD pipeline to
run tests on a wide range of emulated devices.
○ This setup allows them to validate the app's performance on
devices with varying screen sizes and resolutions.

Benefits of continuous testing on parallel devices

Some benefits of continuous testing on parallel devices are:
• Reduced testing time: Parallel execution significantly reduces the
time required for test cycles, enabling faster feedback to developers.
• Enhanced coverage: Testing on various devices ensures
comprehensive coverage, reducing the likelihood of device-specific
issues in the production environment.
• Early bug detection: Parallel testing facilitates the early detection
of bugs, providing developers with timely information for swift
resolution.
 • Cost-efficiency: Cloud-based testing services and emulator farms
offer cost-effective solutions compared to maintaining an extensive
physical device lab.

Conclusion
Android and iOS are the two dominant players in the mobile platform
landscape, each with its own strengths and challenges. Android leads the
global market with its vast ecosystem of manufacturers and developers, but
this diversity results in a fragmented device market that requires a fully
automated DevOps pipeline for effective development. Meanwhile, iOS,
while more controlled due to being produced by a single manufacturer, still
faces challenges related to various device models and operating system
versions. As a result, developing a strong DevOps strategy is crucial for
ensuring seamless updates and user satisfaction on both platforms.
Regardless of the platform, the DevOps approach remains pivotal. Imagine
a situation where, similar to DevOps in web application development,
instead of three major browsers, there were thousands of distinct browser
types. In this case, Automation becomes necessary to ensure any level of
quality assurance. This is precisely why the mobile space focuses on UI test
automation executed on real devices.
Armed with the tools and techniques gained from this chapter, coupled with
a comprehensive understanding of DevOps principles encompassing source
control, build promotion, and security, you are well-positioned to outshine
your peers in Mobile DevOps. This readiness positions you to tackle the
formidable challenge of continuous deployments to millions of Android and
iOS devices globally.

1. Source [Link]
ile-users-since-2010/
2. Source: [Link]
3. Source: [Link]
Join our book’s Discord space
Join the book’s Discord Workspace for Latest updates, Offers, Tech
happenings around the world, New Release and Sessions with the Authors:
[Link]
 Index

A
access control lists (ACLs) 165
ahead-of-time (AOT) compilation 34
Amazon 33
Amazon Web Services (AWS) 92, 268
Android OS fragmentation 286, 287
advanced hardware and 3D support 290, 291
Android for disparate screens 288-290
Ansible 15
Apache Ant 127
terminologies 127-129
Apache Maven 130
terminologies 130-133
API server 103
artifact 151, 152
creating 152-154
artifact publication 164
benefits 165
 best practices 165, 166
to JFrog Artifactory 171, 172
to Maven Central 168-171
to Maven Local 166-168
to Sonatype Nexus repository 171
AWS Serverless Application Model
(SAM) CLI 49
Azure DevOps 137
Azure Kubernetes Service (AKS) 93

B
Bamboo 137
behavior-driven development
(BDD) 137
Berkeley Software Distribution
(BSD) 92
blue/green deployments 234
implementing 235-237

C
Canary deployment 237
Checkmarx 15
Chef 15
Chroot 92
CI/CD pipeline 196
example 275-277
Cloud Native Computing Foundation
(CNCF) 93
code versioning 63
best practices 63
common vulnerability scoring system
(CVSS) 197, 198
components 198, 199
in practice 200
container images
building 213
building, with Eclipse JKube 217-221
managing, by using JIB 213, 214
pushing 213
container providers 97
container runtime 103
Container Runtime Interface (CRI) 93
containers 90, 96
advantages 91
cons 97
evolution 91-93
image terminologies 98, 99
pros 97
versus, virtual machines 94, 98
working 96
continuous build 136, 137
continuous delivery (CD) 263, 264, 271
benefits 265
best practices 271, 272
challenges 272
components 265
evolution 264
 importance 271
versus, continuous deployment 273, 274
continuous deployment 272
best practices 273
challenges 273
importance 272
continuous integration (CI) 120, 121, 264
critical concepts 122, 123
in development team 121, 122
key practices 123-125
continuous updates
need for 266
continuous uptime 267, 268
necessity 268-270
controller manager 103
COPY command 116
cross-site scripting (XSS) 179

D
declarative build script 125-127
build with Apache Ant 127
build with Apache Maven 130
build with Gradle 133
Dekorate 225
for, generating Kubernetes
manifests 225-230
dependency management 154, 155
 with Apache Maven 155-157
with Gradle 157-162
dependency management, in containers 162
basics 162
benefits 162, 163
best practices 163, 164
deployment
Kubernetes deployment 211
planning 211-213
traditional deployment 211
developer 11
DevOps 2, 3
and Microservices 31-33
application development need 7-9
automation 10, 11
benefits 16, 17
business need 5-7
loop 4, 5
need for 5
pillars 4, 5
team roles 11-13
team structure 11
versus, Mobile DevOps 281, 282
DevOps engineer 11-13
DevOps processes 17
application performance monitoring 24
 artifacts repository management 20
build management 19
configuration management 18, 19
continuous delivery 22
continuous deployment 23
continuous integration 22
Infrastructure as code (IaC) 23, 24
release management 20, 21
source code management (SCM) 18
source code review 18
test automation 21
DevOps strategy 13, 14
DevSecOps 180, 181
benefits 182, 183
components 181, 182
DevSecOps pipeline
planning 196, 197
Docker 15, 100
architecture 100
best image building practices 115, 116
container orchestration, with Kubernetes 102
Dockerfile 101
sample 101
Docker image layer 113
Docker on machine 104-111
image layers 113, 114
 tagging and image version management 111, 112
tags, adding to images 112, 113
dynamic application security testing
(DAST) 181, 184, 189
benefits 190, 191
disadvantages 191
working 189, 190

E
Eclipse 82
Eclipse JKube 217, 230
container images, building with 217-221
for generating Kubernetes manifests 230
Elastic Kubernetes Service (EKS) 93
etcd 103
evolving user expectations
meeting 267

F
feature branching workflow 84
forking workflow 85

G
Generate Project button 41
geographically distributed systems
best practices 258, 259
Git 14
Git components 65
Git clients 66
 Git server 66
git-gui 79
Git GUI tools 66
GitHub 14
GitHub actions 137
GitHub desktop 80, 81
gitk 79
GitKraken client 81
GitLab 137
Git tools 69
configuring, to work from
command line 73, 74
Git clients 78-81
Git command-line 69
Git command-line basics 70-73
Git command-line tutorial 77, 78
Git IDE integration 82
Git IDE plug-ins 70
GUI clients 69
initial repository, creating 74-77
local repository, working with 74
Git workflows 83
centralized workflow 83
feature branching 84
forking workflow 85
Git-flow workflow 84, 85
trunk-based development 86
Google Kubernetes Engine
(GKE) 93
GraalVM 34
installing 40
Gradle 133
terminologies 133-135
GraphDriver 110
Group, Artifact, and Version
(GAV) coordinates 49

H
High Availability (HA) 257
HorizontalPodAutoscaler
(HPA) 247
hybrid architecture
characteristics 259

I
infrastructure as code 5
interactive application security testing
(IAST) 193, 194
iOS OS fragmentation
best practices 286
iOS version adoption 285
legacy devices and tailored experiences 285, 286
orchestrated iOS ecosystem 285

J
jails 92
Java Archive (JAR) file 152
java-library plugin 158, 159
Java Virtual Machine (JVM) 34
Jenkins 14, 137, 138
Jenkins pipeline
building 138-140
creating, to perform
SonarQube analysis 205, 206
JFrog Artifactory 171
artifacts, publishing to 171, 172
Jib 213
container images, managing with 213, 214
setting up, in Gradle project 216, 217
setting up, in Maven project 214-216
JUnit 137
Just-in-Time (JIT) compiler 34

K
key performance indicators (KPIs) 196, 253
kubelet 103
kube-proxy 103
Kubernetes 15
for Docker container orchestration 102
master node components 103
running locally 114
worker node components 103
working 102
Kubernetes as a Service (KaaS) 102
Kubernetes deployments 222, 223
 deployment strategy, implementing 231-237
Kubernetes manifests, generating with Dekorate 225-230
Kubernetes manifests, generating with Eclipse JKube 230, 231
local setup 223, 224

L
Let Me Contain That for You (lmctfy) 92
Linux Containers (LXC) 92
logging 253
best practices 246, 247, 253, 254
error tracking 253
performance monitoring 253
LowerDir 110

M
Maven Central 168
artifact, publishing to 169-171
Maven Local repository 166
methods of publishing 166-168
MergedDir 110
metadata 142, 143
capturing 146-148
creating 145
importance 143
insightful metadata, key attributes 144, 145
writing 148-151
Metadata Object Description Schema
(MODS) 145
Micronaut 40-43
Microservices 25
characteristics 27
need for 27
Microservices-based architecture
example 28
Microservices frameworks 33, 34
Micronaut 40-43
Quarkus 44-47
Spring Boot 34-40
Mobile CI/CD pipeline 293
components 293, 294
real-world example 295-297
setting up, in cloud 294, 295
mobile device fragmentation 284
Android OS fragmentation 286, 287
iOS OS fragmentation 285
iOS, versus Android fragmentation 284
Mobile DevOps 279-281
best practices 297-299
challenges 292, 293
continuous testing, on parallel devices 299-301
features 282, 283
versus, DevOps 281, 282
workflow 283, 284
model-view-controller (MVC) 36
monitoring
best practices 246-253
Monolithic 25
migration, to Microservices 29, 30
Monolithic application
analyzing 28
challenges 28
Monolithic architecture 26
characteristics 26, 27
Multicloud architecture
characteristics 260-262
My_Spring_Boot_Application 228

N
National Infrastructure Advisory Council
(NIAC) 198
NetBeans 82
Netflix 33

O
Open Container Initiative (OCI) 92
OpenGL ES 291
open-source tools 15
OpenTelemetry 255
attributes 255
baggage items 255
context propagation 255
span 255
 trace 255
Oracle JDeveloper 83
original equipment manufacturers
(OEMs) 287

P
package management 154
Photoshop Document (PSD) 289
pipeline 138
programmer 11
project object model (POM) 130
Prometheus query (PromQL) 252
pull request
creating 66-69
Puppet 15

Q
quality assurance (QA)
environment 119
quality gates 200
benefits 200, 201
implementing 201, 202
risk management 207, 208
security, implementing with 206, 207
strategies 201
quality management
practical applications 202
with SonarQube 202-204
Quarkus 44-47
R
Rate, Errors, Duration (RED)
methodology 256
reliability engineer 11, 12
Ruby on Rails (RoR) framework 40
runtime application self-protection
(RASP) 194, 195
advantages 195
drawbacks 195

S
Scalable Vector Graphics (SVG) 289
scheduler 103
Scrum Master 11
Secured Shell (SSH) 15
semantic versioning 63
reference 63
serverless architecture 47, 48
setting up 48-57
Service Level Agreement (SLA) 268
service owner 11, 12
Shift-left Security approach 183, 184
benefits 185
challenges 185, 186
key principles 184
Simple Logging Facade for Java
(SLF4J) 254
software development life cycle
(SDLC) 1, 3
software security 177, 178
challenges 180
key elements 179, 180
need for 178, 179
Solaris containers 92
SonarQube 15, 202
Sonatype Nexus repository 171
artifact, publishing to 171
source control management
(SCM) 60, 61
best practices 86, 87
first generation 61
second generation 62
source control, selecting 64, 65
third generation 62
Sourcetree 81
Spring Boot 34-36
static and dynamic security analysis 186
dynamic application security testing 189
static application security testing 186, 187
static application security testing
(SAST) 181, 184-186
benefits 187, 188
disadvantages 188, 189
versus, DAST 191-193
working 187
Substrate VM 34
Subversion (SVN) 83
supply chain security 175
from customer perspective 176
from vendor perspective 175, 176
full impact graph 177
System on Chip (SoC) 290

T
TeamCity 137
Team Foundation Version Control
(TFVC) 65
team lead 11
test automation 137
Test Driven Development (TDD) 122
TestNG 137
tests
maintenance 138
monitoring 138
tracing 255
best practices 255, 256
trunk-based development 86

U
UpperDir 110

V
version control system (VCS) 115, 123
Vertical Pod Autoscaler (VPA) 248
virtual machines 94
components 95
 cons 95, 96
pros 95
working 94, 95
Visual Studio Code 82
vulnerability scoring system 197

W
white-box security testing 186
workload management, in Kubernetes 238, 239
application configuration level 243, 244
health checks, setting up 239-243
Kubernetes deployment configuration level 244, 245
persistent data collections, working with 245, 246
resource quotas, adjusting 243