Batch Processing of

data is a passé
Elevate to Streaming Analytics
for continuous business insights
Index
Executive Summary 3

Introduction 4

Upgrading the data analytics status quo 5

Adapting to real-time analytics 7

Streaming analytics in IoT: the new normal 9

A peek at the future: use cases for streaming analytics 10

Conclusion 12

© Zensar Technologies, 2021 | Page # 2

Executive Summary
Streaming analytics is the latest advancement analytics is the primary source of insights,
in the ever-growing arena of data analytics and where the decision-making onus is on data
its applications. It allows businesses to further that’s current to the millisecond. In a real
the speed and accuracy of critical decisions sense, streaming analytics is already a big part
and actions. Moving away from traditional of everyday use cases such as ride-sharing,
batch processing of data, which does not where instantaneous data processing plays a
quite support proactive and predictive pivotal role. What comes next is the
processes, businesses are fast adopting this as operationalizing of this mode of data
the new paradigm. With the right strategy, capturing and analysis for a wider set of
tools, and resources to equip their teams with, business processes and customer
businesses worldwide can gain a competitive experiences.
edge, as well as stay ahead of the digital
This paper unpacks the growing demand for
transformation curve by implementing
streaming analytics and explores the
new-age technologies such as artificial
real-world applications and feasibility of this
intelligence (AI), machine learning (ML),
phenomenon.
internet of things (IoT), big data, and cloud
infrastructure. For all of these, streaming

© Zensar Technologies, 2021 | Page # 3

Introduction
In today’s big data-powered world, streaming 2027, growing at a CAGR of 26.8% from 2020
analytics is the latest buzzword. It refers to the to 2027. And this change is in no small part
processing and analysis of data records driven by the Internet of Things and the
continuously in real-time, rather than in growth of AI and the accelerated digital
batches. Also known as event stream transformation of businesses after the
processing, this provides a non-stop flow of COVID-19 pandemic. Figure 1 illustrates the
data that can be stored, processed, and acted differences in data flow and architecture
upon. This is especially useful in cases where between batch and stream data processing.
the data generated is low in volume and size, Streaming analytics allows enterprises to
and aids in continuous business insight. understand what is happening now, in
Businesses too are rethinking the way they real-time, by analyzing data from multiple
make decisions, in order to leverage the speed real-time data sources. This can then be
and accuracy that streaming analytics can analyzed to track real-time performance
power. In fact, the global streaming analytics and make instantaneous business
market was valued at $7,740.0 million in 2019 decisions accordingly.
and is projected to reach $52,190.0 million by

Batch Data Processing Architecture

Data transformation and Analytical

integration to a warehouse layer

IoT devices
Data
warehouse
Web/application
ETL
Extraction of data Querying data via
Sensor data
from its sources user interface

Stream Data Processing Architecture

Stream processors perform data Analytical

ingestion and processing layer
IoT devices

Web/application

Sensor data

The birth of data Constant data flow between

as event streams the processor and UI

Figure 1: Batch data processing architecture vs stream data processing architecture

© Zensar Technologies, 2021 | Page # 4

Upgrading the data analytics status quo
Business Intelligence and data-driven and utilized by businesses. This is where
processes have been a staple of every streaming data plays a vital role.
organization for a long time now. However, the
While the world is familiar with use cases of
advent of big data and the addition of an
streaming analytics across real-time location
infinite amount of data sources have made it
tracking, fraud detection, and in the BFSI
challenging to control and streamline the flow
sector for use cases like stock trading,
and structure of data signals. Moreover, every
streaming analytics is not just limited to
business wants to gain a strong data-backed
pattern identification and alerts. It can be
competitive edge, which puts the onus on
integrated into full-scale business analytics to
business intelligence systems to overcome
monitor business performance in real-time. In
these new challenges. Batch processing of
fact, marketing and sales as a segment
data no longer makes the cut. In fact, data
dominated the market in 2019 with a 28.3%
that is more than a few milliseconds old is
share, thanks to the increasing usage of
already considered stale in a lot of industry
streaming analytics in delivering better
use cases, such as tracking in ride-sharing and
customer engagement by employing
hyperlocal delivery services. This makes
customer profiling, campaign management,
real-time corrective action and continuous
demographic analysis and other such
improvement a challenge. It can be likened to
techniques. It can guide businesses in
driving a car while looking at the road behind
proactive decision-making based on business
the vehicle, rather than ahead. The need of
value indicators, as opposed to reactive
the hour is to have a steady stream of truly
course correction based on past data.
current data flowing in, that can be processed

© Zensar Technologies, 2021 | Page # 5

 Consumers
Producers (internal)Services

Rider APP
(internal)Services Mobile App
Driver APP
API/services Real-time, analytics,
Kafka Samza/Flink
Database changelog clusters alerts, dashboards

Etc.
ELK Debugging

Applications
Hadoop
data science

Ad-hoc exploration

Apache
Hive
Analytics reporting

Figure 2: Traditional Streaming Analytics

The above architecture diagram (Figure 2) depicts data signals that have been captured in small
streams using Kafka message broker, and then processed in another layer of Hadoop,
Elasticsearch, and other technologies, and finally passed on to the business application for
insights. With recent developments, however, data can be processed in data streams, and utilized
for insights right as they are captured.

© Zensar Technologies, 2021 | Page # 6

Adapting to real-time analytics
Data collection is only the tip of the analytics iceberg. Making sense of this data to feed into
business processes and business decisions is key. Any company that deals with big data could
find it advantageous to implement streaming analytics for continuous, real-time data processing.
For example, retail inventory data gets processed in batches to track revenue and sales patterns.
Data from the store, warehouse, and supply chain feed into this analysis. Imagine being able to see
this data in real-time, as moving inventory or supply chain activity, for example. With this data, it
would be possible to make adjustments or changes to any touchpoint in the supply chain, rather
than make reactive decisions later.

While this may sound ideal, businesses need to first identify KPIs that make the most sense for
them to track. Also important is to acknowledge data sources that cannot be tracked in real-time,
and plan around these gaps.

Using the same retail example as above, these data streams could be:

Real-time inventory Revenue metrics, to Coupon codes and offers

movement, to track what discover actual speed of redeemed, to get a sense of
times of the day are busy meeting business targets which ones are performing
and what parts of the and planning future targets well, what needs to be
retail catalog are popular promoted, and what needs
among customers to be rebalanced

Other use cases where this level of granular data streaming would be critical are real-time
personalization of app or web experiences, anomaly and fraud detection in online transaction
systems, real-time health monitoring, and proactive healthcare.

With about 90% of companies believing that investing in real-time data technologies will be the
key to remaining competitive, the case for this shift is made. But 66% find it hard to reduce time to
value from their data, due to culture and the lack of people and skills. Becoming a streaming
analytics-led business is a large-scale change in strategy, and it is important to start small and
flexible.

© Zensar Technologies, 2021 | Page # 7

The first step is identifying real-time business indicators and near-real-time business indicators
that are mission-critical. Examples of real-time indicators are the number of SKUs sold, the location
of the ride-share driver, revenue generated, and media being consumed on entertainment and
social media platforms. Near-real-time indicators could look like ticket inventory on online ticketing
platforms, and payroll processing data. In essence, these indicators might be momentary, but
upon aggregating, they serve to guide major business decisions. Monitoring and analyzing
streaming data can greatly improve operational efficiency and shed light on business challenges
and opportunities.

© Zensar Technologies, 2021 | Page # 8

Streaming analytics in IoT: the new normal

Device/sensor analytics Edge analytics Core analytics

Gateway Data flow Cloud

Data
center

Figure 3: The flow of data in streaming analytics

The above architecture diagram (Figure 3) depicts the flow of real-time business indicators that
help in monitoring and anticipating the realization of business targets. This continuous flow of
data can help prevent untoward situations and enable businesses to take corrective action.
• With the use of streaming analytics, as depicted above, patterns and relationships can be
identified in the data extracted from multiple input sources — such as devices and internal
systems — across various business functions, applications, and more.
• These patterns can be used to trigger actions and initiate workflows such as creating alerts,
feeding information into a reporting tool, or storing transformed data for later use. Today, there
are open-source bundles and COTS cloud tools that can help businesses perform these actions.

© Zensar Technologies, 2021 | Page # 9

A peek at the future:
use cases for streaming analytics
Connected Devices (IoT)
Monitor usage and activity on consumer devices to raise automated requests or
alarms. In business-facing devices, this can reduce latency and streamline inventory
and supply chain.

• Connected homes: Automated lighting and appliance shutoff, as well as intruder

detection and alerts that can enable the homeowner to trigger safety measures.
• Connected automobiles: Automation of navigation features and real-time traffic
monitoring to optimize driving routes.
• Consumer IoT: Automatic triggers to place purchase orders or maintenance
requests based on data collected.
• Wearable technology: Streamed monitoring of vitals and proactive alerts on
fitness and healthcare measures.
• Enterprise IoT: Real-time product lifecycle management and strategy mapping.
• Industrial IoT: Real-time monitoring of machinery for proactive maintenance and
prevention of wear-and-tear.

Financial services
Monitor account activity to detect anomalies and generate a security alert for
abnormal activity or usage.

Investment services
Track market changes and adjust customer portfolios based on configured
constraints, such as selling when a certain stock value is reached.

© Zensar Technologies, 2021 | Page # 10

E-commerce
Analyze user clickstream data to optimize and personalize the shopping experience
with real-time pricing, promotions, product recommendations.

Media and entertainment

Real-time analysis of video data for object tracking and understanding audience
interaction, to provide more personalized experiences.

Retail
Analyzing real-time POS data, and matching it with purchase history, customer
behavior, and loyalty card performance to improve customer experience.

Cybersecurity
Monitor online activity and trigger cybersecurity alarms when certain rules have been
breached or frauds committed. This could be especially useful to social media
platforms.

© Zensar Technologies, 2021 | Page # 11

Conclusion
Streaming analytics is becoming the new paradigm, with businesses being able to leverage
continuous, real-time data, agnostic of industry. While traditional business analytics may have its
benefits and a stronghold over legacy business processes, streaming analytics is fast becoming
the way of the future. The need for real-time decision-making has made it absolutely necessary for
firms to turn to streaming data for advanced insights, in order to stay truly competitive. Knowing
that all data is current and accurate can be a great way to boost confidence levels in business
decisions.

Getting started with streaming analytics is not just a passing trend. This could be a crucial movie
for a business that defines their growth for the next decade. The key is to develop a solid business
case for the shift and work with a realistic and agile plan. Going live with a hybrid model and
periodically reviewing the system will help operationalize this landmark phenomenon in data
analytics.

© Zensar Technologies, 2021 | Page # 12

Sanjiv Kumar Sajeed Kazi
Sr. Solution Architect – Analytics Practice Sr. Solution Architect – Analytics Practice
[Link]@[Link] [Link]@[Link]

We conceptualize, build, and manage digital products through experience design, data
engineering, and advanced analytics for over 130 leading companies. Our solutions
leverage industry-leading platforms to help our clients be competitive, agile, and
disruptive while moving with velocity through change and opportunity.

With headquarters in Pune, India, our 10,000+ associates work across 33 locations,
including San Jose, Seattle, Princeton, Cape Town, London, Singapore, and Mexico City.

For more information please contact: velocity@[Link] | [Link]