Chapter Two

LITERATURE REVIEW

2.1 Conceptual Review

The rise of abnormal human behavior detection systems, especially for real-time surveillance,

has transformed security measures in urban environments. With increasing security challenges,

such as terrorist attacks and violent crimes, intelligent systems leveraging computer vision and

machine learning have become indispensable. This chapter reviews the conceptual, theoretical,

and methodological foundations of such systems, highlighting key contributions and identifying

research gaps.

Human Action Recognition (HAR) is a vital technology with numerous applications that impact

daily life, including video surveillance systems used in smart buildings, banks, and parking lots.

HAR enables effective communication between humans and machines through techniques like

hand gesture classification and speech recognition. The analysis of video frames from security

cameras to detect and control abnormal behavior forms the foundation of automated monitoring

systems. These systems are particularly crucial for monitoring elderly or sick individuals who

live alone, as they require constant and reliable oversight. Effective action detection depends on

factors like environmental context, detection time, and behavior specificity. The process of HAR

involves several key stages: preprocessing to separate foreground from background and track

individuals across video frames, feature extraction to identify meaningful patterns, and action

classification to categorize behaviors. Common methods, such as background subtraction, detect

motion by comparing video frames with a static background, but challenges like cluttered

environments and viewpoint variations make accurate recognition difficult. Traditional HAR
approaches rely on handcrafted features and pattern recognition techniques but often struggle

with real-time applications due to their dependency on specific features. To address these

challenges, the proposed Smart Deep Learning-Based Human Behavior Classification (SDL-

HBC) model integrates advanced techniques for real-time video surveillance. The SDL-HBC

model employs adaptive median filtering (AMF) for noise reduction during preprocessing,

utilizes Capsule Networks (CapsNet) for robust feature extraction with hyperparameter tuning

via the Adam optimizer, and applies Differential Evolution (DE) with Stacked Autoencoder

(SAE) for accurate action classification. Validated using benchmark datasets such as the KTH

dataset, the SDL-HBC model demonstrates superior performance in detecting and classifying

human actions, even in challenging scenarios. By addressing limitations in traditional HAR

methods, this model offers an intelligent, efficient solution for smart surveillance applications,

improving automated care and public security monitoring1.

Modern video surveillance systems have evolved beyond traditional recording for offline

forensic analysis to intelligent solutions capable of proactive incident prevention. These systems

combine computer vision, pattern recognition, and cloud computing to process and analyze video

data in real-time, reducing the reliance on human operators. They find applications in public

safety, such as detecting emergencies, monitoring traffic, and analyzing human activity. Enabled

by advancements in processors, sensors, and smart cameras, these systems can classify behaviors

as normal or abnormal, allowing for rapid alerts and decision-making. For instance, near critical

infrastructure like ATMs, intelligent surveillance systems can detect and report suspicious

behaviors, improving response times and overall security2.

Modern intelligent video surveillance systems primarily focus on areas such as virtual reality,

and video stitching. Abnormal detection holds significant importance due to its critical

application scenarios and has garnered considerable attention over the past two decades 3.

The global crime rate was escalating, with gun violence, terrorist activities, burglary and terrorist

activities becoming increasingly prevalent and dangerous. Addressing such threats requires

effective countermeasures. Human behavior recognition involves analyzing video footage to

determine actions and classify them into predefined categories, such as normal or abnormal.

Techniques employed include:

1. Hidden Markov Models (HMMs): These models represent human activities using

postures and velocity vectors of body parts. By analyzing the position and movement of

limbs, these models can classify actions like walking or running.

2. Blob-based Learning: This approach leverages the shape and motion of objects (blobs)

in video frames to identify abnormal behaviors, such as sudden aggressive movements.

3. Crowd Behavior Models: Inspired by sociology, these models simulate crowd behavior

in specific environments, examining the interactions between individuals to detect

emergent patterns, such as panic or congestion.

The increasing elderly population has heightened the need for systems that ensure their safety.

Abnormal behavior detection systems based on wireless sensor networks have been proposed for

assisted living environments. These systems monitor individuals, compare episodes of behavior

using spatial and temporal information, and use similarity scoring functions to identify

potentially harmful deviations. By analyzing CCTV footage, authorities can monitor, detect, and

assess situations effectively4.

The rapid growth of IoT has increased the number of smart home devices, expected to reach 5

billion by 2025. However, this expansion introduces risks due to abnormal behaviors, which

stem from malicious attacks (e.g., hackers exploiting vulnerabilities to disable Security cameras).

Deep learning has been applied to address these issues by modeling user behavior and detecting

anomalies. Models like DeepMove (using RNNs), SASRec (using self-attention), and

transformer-based architectures effectively analyze sequential behavior data. Despite their

success, these models face challenges in smart homes, including handling diverse behaviors,

meeting real-time processing needs, and ensuring data privacy. Developing tailored solutions that

address these challenges is crucial to enhancing smart home security and reliability 5.

Deep Neural Networks (DNNs) have been extensively utilized in machine learning for pattern

recognition, both independently and in conjunction with Convolutional Neural Networks (CNNs)

and Recurrent Neural Networks (RNNs). For instance, DNNs have been applied to Human

Activity Recognition (HAR) by leveraging sensor-derived features. In another study, Principal

Component Analysis (PCA) was employed for feature selection, followed by DNN-based

activity learning. Furthermore, research has shown that with sufficiently deep architectures and

large datasets, activities can be identified directly from raw sensor data, bypassing the need for

manual feature engineering. This underscores the versatility of DNNs in adapting to diverse data

modalities and tasks6.

Deep Convolutional Neural Networks (CNNs) are employed for gesture classification, achieving

high accuracy through transfer learning a method that leverages pre-trained models to save

resources and improve performance. The proposed system also emphasizes robust preprocessing

of video data under varying conditions such as distance and lighting, ensuring reliability in real-

world scenarios. This integration of modern video capture devices, computer vision, and deep
learning showcases the potential for intelligent surveillance to enhance safety and efficiency in

urban environments2.

Abnormal detection in videos plays a vital role in ensuring security across various settings, both

indoors and outdoors, including campuses, waiting halls, and shopping malls. Additionally, the

rarity and infrequency of abnormal events make real-time supervision even more demanding and

complex3.

Unlike traditional systems, intelligent surveillance can analyze and interpret video data in real

time, enabling the detection and recognition of abnormal behavior efficiently while reducing

human intervention. Research in this field focuses on video intelligent analysis to quickly and

accurately identify unusual actions occurring in various environments, such as theft, fighting, or

dangerous behaviors in schools and nursing homes. These systems provide early warnings for

incidents like robbery, wrestling, or pushing, helping to prevent harm to vulnerable populations

such as children and the elderly. However, challenges remain, including the lack of universal

recognition algorithms, high misjudgment rates, and the computational complexity of existing

methods. Traditional approaches like template matching are limited to simple scenes, while deep

learning-based spatiotemporal feature algorithms improve detection but suffer from slow

execution due to steps like optical flow computation. Pose estimation methods, affected by

target-background interactions, also underperform in complex scenarios. To address these issues,

a novel approach involving a small dataset of pedestrian anomalous behaviors, including actions

like falling, boxing, and walking, is proposed. By integrating multiscale feature fusion and

single-scale prediction into a three-dimensional (3D) convolutional network, this method aims to

improve the accuracy and efficiency of abnormal behavior detection in intelligent video

surveillance systems7.
Object tracking is another crucial step in suspicious action detection. It involves tracing an

object's position over sequential frames to create a trajectory. Object tracking uses

representations like object contours, geometric shapes, or articulated points. However, challenges

such as noise, complex object shapes, and partial occlusion of objects often complicate this task 8.

2.2 Methodological Review

To achieve accurate user behavior sequence anomaly detection in smart homes, we propose

Smart Guard, a system designed to address the limitations of existing methods. The workflow of

Smart Guard involves several key components. During the training phase, the Loss-guided

Dynamic Mask Strategy is used to mask hard-to-learn behaviors based on the loss vector from

the previous epoch. This is followed by the application of the Three-level Time-aware Positional

Encoder, which captures temporal information, such as order-level, moment-level, and duration-

level details of the behaviors. This information is then combined with the device control

embedding to form the behavior embedding. The behavior embedding is subsequently processed

by an attention-based encoder-decoder architecture to extract contextual information and

reconstruct the source sequence. In the inference phase, the Noise-aware Weighted

Reconstruction Loss is utilized to assign different weights to behaviors based on the loss vector

from the training data, ultimately calculating a reconstruction loss score. If this score exceeds a

defined threshold, Smart Guard triggers an alarm. The Loss-guided Dynamic Mask Strategy

leverages the strengths of masked autoencoders to focus on learning hard-to-learn behaviors that

occur rarely, thereby improving reconstruction accuracy. Unlike traditional autoencoders, which

aim to reconstruct the entire input or randomly masked portions, Smart Guard introduces a top-k

loss mask strategy that targets behaviors with higher reconstruction loss. Initially, the model is
trained without masking to accelerate convergence by learning relatively easy tasks.

Subsequently, the top-k loss mask strategy is employed to emphasize the learning of harder-to-

learn behaviors, dynamically updating the loss vector at each epoch to guide the mask strategy.

Experimental results on the SP dataset demonstrate that this strategy reduces variance in

reconstruction loss and improves model performance. To incorporate temporal information in

user behavior sequences, Smart Guard employs a Three-level Time-aware Positional Encoder.

This encoder captures absolute and relative timing details, such as the order of behaviors, the

hour and day of behavior occurrence, and the duration between consecutive actions on the same

device. These temporal features are combined into a positional embedding and added to the

device control embedding to produce the final behavior embedding. The sequence encoder, based

on a transformer architecture, extracts global semantic information by leveraging multi-head

attention and position-wise feed-forward networks. The sequence decoder, mirroring the

encoder’s structure, reconstructs the input sequence by predicting the probabilities of each

behavior. The reconstruction loss is optimized using a cross-entropy objective function, which

adapts based on whether masking is applied during training. Finally, the Noise-aware Weighted

Reconstruction Loss is introduced to handle challenging noise behaviors during the inference

phase. By assigning different weights to behaviors according to their loss values, this mechanism

ensures that Smart Guard prioritizes meaningful anomalies while reducing the impact of noise.

Together, these components enable Smart Guard to effectively detect anomalies in real time,

making it a robust solution for ensuring the safety and security of smart home environments5.

Foreground object detection is the initial and critical step for identifying suspicious activities.

Moving objects are treated as foreground, while static objects form the video background.
Background subtraction is a key technique used to differentiate the current frame from the

background model to detect moving objects. Early approaches relied on single Gaussian models,

which later advanced to multimodal Gaussian distributions, such as the Mixture of Gaussians.

Dual background modeling is also applied to detect abandoned objects. However, challenges

such as noise, shadows, and illumination effects can lead to false classifications. To address these

issues, researchers use techniques like color normalization, Gaussian smoothing, radial reach

filters, and fuzzy histogram color normalization. Object tracking is another crucial step in

suspicious action detection. It involves tracing an object's position over sequential frames to

create a trajectory. Object tracking uses representations like object contours, geometric shapes, or

articulated points. However, challenges such as noise, complex object shapes, and partial

occlusion of objects often complicate this task. Feature extraction is an essential step in detecting

suspicious activities from video footage. Selecting the right features is critical for ensuring

accurate detection and classification of activities. Robust feature selection directly impacts the

success of the detection framework. Finally, classification is performed to distinguish between

normal and abnormal behaviors after extracting foreground objects. Researchers employ various

classification methods to achieve this, including Support Vector Machines (SVM), K-Nearest

Neighbors (K-NN), Neural Networks, and Multi-SVM. These methods are widely used to

classify activities effectively and form a key component of intelligent surveillance systems 8.

Supervised methods aim to model normal and abnormal behaviors using labeled data and are

typically designed to detect specific abnormal behaviors predefined during the training phase,

such as fighting detection, loitering detection, and falling detection. Many supervised methods,

including the Bag of Words (BOW) approach, are commonly used in literature for event

detection in videos. The BOW approach represents each video or frame with a histogram of
words, typically involving local image features or trajectories. A dictionary of words is created,

and the histogram counts the frequency of each word from the dictionary in the video. This

approach is often paired with support vector machine (SVM) classifiers, which are particularly

effective for detecting aggressive behaviors and recognizing crowd anomalies. An algorithm for

abnormal behavior recognition based on human body part estimation using geodesic graphs and

an SVM classifier was introduced. However, its performance is heavily influenced by the

accuracy of detecting the human body zone, especially when individuals are close to each other.

Semi-supervised methods require only normal video data for training and can be categorized into

rule-based and model-based approaches. Rule-based methods aim to develop a rule based on

normal patterns, considering any sample that does not fit this rule as an outlier or anomaly. Lu,

Shi, and Jia proposed a rule-based method using sparse coding to detect abnormal behaviors.

Although this approach provided good results with a short execution time (150 frames per

second), its performance was highly affected by the threshold value. Model-based semi-

supervised methods focus on building a model representing normal behaviors, and any new

sample that does not align with this model is classified as an anomaly. These models are

relatively easy to generate and interpret, but they are sensitive to multiple parameters and may

result in false alarms when unknown normal data is incorrectly identified as abnormal.

Unsupervised methods, on the other hand, rely on statistical properties extracted from unlabeled

data. These methods are fast and easy to perform, with no prior knowledge required. However,

they are time-consuming when it comes to interpreting results. One major assumption in

unsupervised methods is that abnormal behaviors are rare compared to normal behaviors, which

might not always hold true9.

Transfer learning is a machine learning technique that leverages pre-trained models from one

task to improve performance on a related task, thus reducing the need for large datasets and long

training periods. By utilizing the knowledge gained from the initial training, usually on vast

datasets, transfer learning enables models to generalize better to new tasks with fewer labeled

examples. Typically, this process involves fine-tuning a pre-existing model on a smaller, task-

specific dataset, allowing it to adapt to new features while retaining the beneficial representations

learned during the initial training. This approach is particularly useful in domains with limited

labeled data, such as medical imaging or natural language processing, where acquiring annotated

data is both challenging and resource intensive. Convolutional Neural Networks (CNNs) are

deep learning architectures designed for visual data, such as images. They consist of multiple

layers that automatically learn and extract features from input data. The key component of CNNs

is the convolution layer, which uses small filters that slide over the image to extract crucial

features like edges and textures while maintaining the spatial relationships between pixels.

Initially, the network learns simple features, and as data moves through deeper layers, it captures

more complex features, enabling accurate object and pattern recognition. CNNs also include a

down sampling layer known as pooling, which reduces the size of feature maps produced by the

convolution layers, making the model smaller, faster, and more efficient without losing essential

information. Max pooling is a common technique used in which the maximum value from a set

of local pixels is selected. After convolution and pooling layers, CNNs typically have fully

connected layers that combine the learned features to make predictions or classifications. This

architecture makes CNNs powerful tools for various tasks such as image classification, object

detection, and key point detection. For video data, 3D convolution, or 3D ConvNet, is

specifically designed to capture the temporal dimension in addition to the usual spatial
dimensions. While 2D convolution processes single images, 3D convolution adds a temporal

dimension, making it ideal for action recognition and video classification, where dynamic

changes over time must be detected. After applying 3D convolution, a batch normalization layer

is used to normalize the inputs in each mini-batch, ensuring the data distribution across mini-

batches is consistent. This helps reduce internal covariate shift and speeds up the training

process. The normalized data is then passed through the ReLU (Rectified Linear Unit) activation

function, which prevents the vanishing gradient problem by enabling effective learning,

particularly in deeper networks. ReLU transforms negative values to zero and leaves positive

values unchanged, introducing non-linearity without adding significant computational costs. The

combination of 3D convolution, batch normalization, and ReLU activation enhances the model's

ability to learn complex features from video data, resulting in higher accuracy for tasks that

involve dynamic content10.

The system architecture for intelligent video surveillance integrates various hardware and

software components to efficiently capture, process, analyze, and store video data. The primary

objective is to create a system capable of automatically detecting, analyzing, and responding to

events of interest in real-time. One of the key functions of intelligent video surveillance is to

generate alerts when suspicious activity is detected, enhancing security and enabling timely

responses. The design of the system architecture will depend on the specific requirements of the

application, ensuring that the surveillance system meets the desired objectives and functions

effectively in the intended environment. The term "Three-Dimensional Network" (3D) often

refers to a network that employs two-dimensional (2D) layers. This process helps determine the

low-level feature representations, generating 3D volume spaces such as cubes or cuboids. These

3D features are particularly useful in identifying specific occurrences in video footage. In a 3D

network, spatial-temporal convolutions are performed by convolving a 3D kernel with the space-

time volume. These convolutions effectively capture motion data by correlating the feature maps

generated by the convolution layers with the sequence of frames produced by subsequent layers.

Recurrent Neural Networks (RNNs) are a type of artificial neural network that can process a

sequence of inputs while maintaining their state throughout the sequence. This makes them

highly valuable for deep learning tasks, as traditional feed-forward neural networks do not

consider the order of inputs and cannot store previous information. By recording the output of a

particular layer and feeding it back into the input, RNNs can anticipate the work of that layer.

This process compresses the nodes of numerous layers into a single recurrent layer, RNNs were

created to address several limitations of feed-forward neural networks, including the inability to

process sequential data, the focus on only the most recent incoming data, and the lack of memory

for previous inputs. RNNs overcome these limitations by retaining an internal memory, which

enables them to recall prior inputs. They are commonly used in sequence-based tasks, sentiment

analysis, video classification, and named entity recognition for sequence labeling11.

The basic model of HAR involves essential steps such as preprocessing, motion detection, data

filtration, feature extraction, and classification. These steps are common in most supervised

machine learning algorithms and are fundamental in ensuring that the system performs

efficiently. Preprocessing plays a critical role by cleaning the raw data, making it more suitable

for machine learning tasks. This stage focuses on eliminating noise, handling missing data, and

correcting any inconsistencies. The motion detection step identifies areas of movement within

the video data, which is essential for tracking human activities. Image filtration further refines

the data by removing irrelevant features or noise, which could otherwise hinder the performance

of the model.
Once the data is cleaned and relevant features are identified, feature extraction is employed.

This process reduces the dataset's dimensionality by selecting the most significant features that

best represent the underlying human activities. These features help the system focus on important

aspects, thus improving efficiency and performance. Following feature extraction, the system

uses classification algorithms to categorize the activities as either normal or abnormal. This is

where machine learning models, particularly those based on deep learning, come into play.

Classification models like K-Nearest Neighbor (KNN), Support Vector Machine (SVM), and

more advanced deep learning algorithms such as Convolutional Neural Networks (CNN) and

Recurrent Neural Networks (RNN) are applied. These models have significantly improved the

accuracy of HAR systems by handling more complex and varied data.

In addition to these stages, the process also involves training and testing the model. The training

set is used to teach the model how to recognize various human activities, while the testing set

helps assess the model's ability to generalize and detect activities in new, unseen data. Alarm

systems are integrated into the architecture to notify security personnel when abnormal behavior

is detected, enhancing the system's real-time responsiveness.

System Architecture: The system architecture of an intelligent video surveillance system for

HAR involves several interconnected components that work together to capture, process,

analyze, and classify video data:

1. CCTV Camera: The starting point of the system, where video footage is captured. The

camera monitors the environment and records human activities in real-time.

2. Preprocessing: Raw video data is processed to eliminate noise, fill in missing values, and

correct inconsistencies. This step ensures that the data is ready for the subsequent

analysis.

3. Motion Detection: The system identifies movement within the video frames. This helps

to isolate the regions where human activity is occurring, allowing the system to focus on

areas of interest.

4. Image Filtration: After motion detection, irrelevant or extraneous data is filtered out to

enhance the quality of the input data, making it more suitable for feature extraction and

classification.

5. Feature Extraction: In this stage, the system identifies key features in the video data,

such as the shape, motion patterns, or behavior of the people in the footage. This reduces

the complexity of the data by selecting only the most relevant attributes for further

analysis.

6. Classification: Machine learning algorithms classify the extracted features as normal or

abnormal activities. This classification is crucial in identifying whether any suspicious

activity has taken place.

7. Training & Testing: The system uses labeled datasets to train machine learning models.

The model is trained using a training set and then evaluated on a testing set to assess its

performance and accuracy.

8. Alarm: If the system detects abnormal behavior, an alert or alarm is triggered, notifying
security personnel of the potential threat. This step allows for real-time monitoring and
quick responses to security concerns.
Development in the field of HAR and anomaly detection for surveillance using supervised

learning, especially deep learning methods has proven to be ground-breaking. Supervised

learning conceivably is severed into an individual class or multi-class problems. The

development of Human Activity Recognition (HAR) systems has significantly benefited from

Convolutional Neural Networks (CNN), which are well-suited for recognizing complex human

gestures and activities. CNNs automate feature extraction and can handle complex, nonlinear

relationships in time-domain data, making them efficient for HAR tasks. They have advantages

such as sparse interaction, equivariant representation, and parameter sharing, which help improve

memory usage and runtime efficiency. Various CNN-based models have achieved impressive

recognition rates, with some achieving up to 99% accuracy in CCTV-based HAR tasks.

Recurrent Neural Networks (RNNs) are also widely used for HAR, especially for recognizing

time-varying patterns in sequential data. Basic RNNs struggle with long-term dependencies due

to the vanishing gradient problem, which is addressed by Long Short-Term Memory (LSTM)

networks. LSTM-based RNNs are better suited for capturing temporal dependencies and are

often used in combination with CNNs for enhanced performance. Hybrid architectures that

combine CNNs for feature extraction and LSTMs for temporal learning have been proposed to

improve real-time performance and accuracy. These models have achieved better results in

detecting abnormal activities and handling class imbalances in HAR tasks. Additionally,

integrating Extreme Learning Machines (ELM) as classifiers has further improved the efficiency

and real-time capabilities of these hybrid models.

Overall, while there have been significant advancements in HAR, challenges such as class

variety, real-time performance, and computational complexity remain. Hybrid approaches

combining CNNs, LSTMs, and ELMs are seen as a promising solution for addressing these

issues and improving HAR systems in the future12.

The system integrates multiple components to achieve real-time detection and recognition of

abnormal behaviors in intelligent surveillance systems. The core architecture employs a camera

array consisting of PTZ (pan-tilt-zoom) cameras, which are divided into two categories: large-

area cameras and local-area cameras. Large-area cameras are used to monitor a broad field of

view, while local-area cameras are focused on capturing finer details, such as facial features. The

video feeds from these cameras are processed by subsystems designed for change detection, face

recognition, behavior recognition, and semantic situation generation. Below is a detailed

breakdown of the components and implementation. The change detection subsystem is the first

module in the system. It processes video feeds to identify targets such as people or vehicles. For

this, moving objects in the scene are detected using background subtraction or advanced object

detection algorithms like YOLO (You Only Look Once). YOLO is a pre-trained Convolutional

Neural Network (CNN) capable of detecting 88 object classes from the COCO dataset. When a

person is detected, the system uses the local-area PTZ camera to zoom in and capture the

person's face. In contrast, for detecting vehicles, the system analyzes their motion or static

behavior, such as illegal parking. Motion behavior recognition is a critical subsystem that tracks

moving targets and analyzes their trajectories to recognize normal and abnormal behaviors.

Object Tracking: The tracking process involves template matching, which compares

detected objects across consecutive frames to output their trajectory over time. This

trajectory includes the object's center coordinates and dimensions of its bounding box.

Short-term and Long-term Motion Recognition

  Short-term motion recognition is performed over 4-second intervals to capture

immediate actions like sudden gestures or quick movements.

 Long-term motion recognition is conducted over 10-20 seconds to analyze

extended behaviors, such as walking across an area or loitering.

Motion Database and CNN Training: The motion patterns, extracted from trajectories, are

preprocessed with dilation and erosion operations to enhance clarity. These patterns are

then stored in a motion database, which serves as input for training a CNN model to

classify behaviors as normal (e.g., walking, standing) or abnormal (e.g., running, falling,

fighting).

Face Recognition: Once a human target is detected, the face recognition subsystem

performs the following steps:

Face Detection: The system uses the Multi-task Cascaded Convolutional Network

(MTCNN) for detecting faces. This model integrates face detection and facial alignment

to ensure high accuracy, even in varying lighting conditions or partial occlusions.

Face Feature Extraction: The system employs the VGGFace2 model, a deep CNN trained

on 3.31 million face images spanning 9,131 identities. Face images are resized to

224x224 pixels, and the model generates 2,048-dimensional embeddings that represent

unique facial features. These embeddings are then normalized, and multiple embeddings

of the same individual are averaged to create a standardized representation for that

person.

Face Recognition Process Once the face embeddings are extracted, the system computes

the cosine similarity between the input face embeddings and the stored embeddings in the
face database. If the similarity exceeds a defined threshold, the face is identified as a

known individual. If not, the person is labeled as unknown, and their face is added to the

database for future recognition13.

The Abnormality Converging Scene Analysis Method (ACSAM) is a comprehensive

approach designed to detect abnormal behavior in crowded scenes using Convolutional

Neural Networks (CNNs) and textural features. The method integrates image processing

techniques with deep learning to identify anomalies in real-time, making it suitable for

smart surveillance systems. The process begins with data acquisition, where images of

crowded scenes are captured using surveillance cameras or imaging devices. These

images are then preprocessed to remove noise, enhance quality, and normalize the data,

ensuring they are ready for feature extraction. The preprocessing stage is critical as it lays

the foundation for accurate analysis and detection. The core of the ACSAM method lies

in feature extraction, where textural features such as homogeneity, contrast,

and regularity are extracted from the preprocessed images. These features are essential

for identifying patterns in crowd behavior. Additionally, sequence-based features are used

to capture temporal patterns from video frames, enabling the analysis of dynamic

behavior in crowded scenes. The extracted features are then fed into a Convolutional

Neural Network (CNN), which processes the data through multiple layers to identify

complex patterns. The CNN categorizes scenes into individual and crowd behaviors and

differentiates between normal and abnormal activities. This categorization is achieved

through specialized training layers, including the Abnormality Training Layer and

the Crowd Behavior Training Layer, which are designed to detect anomalies at both

individual and collective levels.

The Abnormality Training Layer focuses on identifying abnormal behavior at the

individual level by comparing textural features and detecting deviations from established

patterns. On the other hand, the Crowd Behavior Training Layer analyzes collective

behavior patterns to detect anomalies in crowd dynamics. These training layers play a

crucial role in improving the accuracy and reliability of the system. The CNN

outputs abnormal factors, which indicate potential anomalies in the crowd scenes. These

factors are analyzed to detect deviations from normal behavior, and low convergence in

abnormal factors suggests inconsistencies that require further investigation. The final step

involves identifying abnormal behavior based on the analyzed data, achieved by

comparing the current and previous states of the scene. The ACSAM method is supported

by several mathematical formulations that formalize the processes of feature extraction,

categorization, and training. One of the key strengths of the ACSAM method is its

integration of deep learning techniques, which allow for automated feature extraction and

improved accuracy in detecting complex patterns. The method is also designed to handle

real-time data, making it suitable for smart surveillance systems. By combining spatial

(textural features) and temporal (sequence-based features) analysis, the method provides

a holistic approach to anomaly detection. However, there are some limitations and

challenges. The use of deep learning models, especially CNNs, requires significant

computational resources, which may limit deployment in resource-constrained

environments. Additionally, the method's performance may vary across different datasets

and environments, requiring fine-tuning for specific use cases. Supervised learning

approaches also rely on large labeled datasets, which can be time-consuming and

expensive to create. The ACSAM method has several applications, particularly in smart
surveillance systems for detecting abnormal behavior in crowded public spaces such as

airports, stadiums, and urban areas. It can also enhance security on university campuses

by identifying suspicious activities and monitor large-scale events for potential security

threats or emergencies. Future directions for this method include optimizing

computational complexity through techniques such as quantization and pruning,

exploring unsupervised learning approaches to reduce reliance on labeled data, and

incorporating additional data sources (e.g., audio, sensor data) to improve detection

accuracy. The ACSAM method presents a robust framework for abnormal behavior

detection in crowd scenes using CNNs and textural features. By addressing key

challenges such as computational complexity and generalization, this approach has the

potential to significantly enhance the capabilities of smart surveillance systems. Future

work should focus on optimizing the method for real-time deployment and exploring

unsupervised learning techniques to further improve its applicability. This method

represents a significant step forward in the field of real-time anomaly detection, offering

a reliable and efficient solution for monitoring crowded environments14.

Abnormal behaviors are categorized into various types, including violence, panic,

loitering, falling, abuse, patient monitoring, and elderly care, based on factors such as

gestures, pose variations, body movements, and interactions between individuals. Deep

learning technologies, particularly Convolutional Neural Networks (CNNs), have been

widely explored to recognize these behaviors from recorded video data. This research

paper delves into the recognition of abnormal behaviors using various CNN-based

models, evaluating their performance across different datasets, input image sizes, and

metrics, while also identifying gaps in existing frameworks. Loitering refers to

individuals lingering in public spaces without a clear purpose for an extended period.

This behavior can be indicative of illegal activities such as theft or drug dealing, or it may

be related to medical conditions like Alzheimer's disease. Loitering detection models

often rely on trajectory analysis and sequential patterns to identify repeated movements,

such as stopping, walking, or turning. For instance, the Generalized Sequential Patterns

(GSP) approach has been used to detect loitering by identifying micro-patterns in human

motion. Other methods, such as Gaussian Mixture Models (GMM) and Manshift, have

been employed for object tracking and trajectory classification. However, these models

often require manual preprocessing and parameter tuning, which can be time-consuming.

More advanced approaches, such as 3D CNNs, have shown promise in learning

spatiotemporal features for loitering detection, achieving high accuracy but struggling

with long-distance and overlapping objects. Falling is a critical abnormal behavior,

especially for elderly individuals. Depth-sensor-based models and background

subtraction techniques have been used to detect falls by analyzing shape changes and

motion patterns. For example, the Curvature Scale Space (CSS) feature extractor

combined with Support Vector Machines (SVM) has been employed to recognize falling

actions. However, these methods often fail to handle occlusions and overlapping objects.

More recent approaches, such as optical flow-based CNNs and multi-stream CNNs, have

improved fall detection accuracy by leveraging RGB and depth data. Despite their

success, these models are computationally expensive and struggle with multi-person

scenarios. Monitoring patients for abnormal behaviors, such as falls, chest pain, or

vomiting, is essential in healthcare settings. YOLO-based CNN architectures have been

proposed to detect multiple abnormal activities in real-time. These models perform well
in uncrowded scenarios but struggle with overlapping objects and multi-human detection.

Similarly, VGGNet-based models have achieved high accuracy in fall detection but face

challenges in generalizing across different datasets and environments. Violence detection

involves recognizing actions such as punching, kicking, and fighting. Traditional

methods, such as GMM with fuzzy clustering, have been used to classify violent

behaviors but require manual parameter computation. More advanced approaches, such

as 3D CNNs and LSTM-based models, have shown superior performance by learning

spatiotemporal features from video data. For instance, Wide Dense Residual

blocks combined with LSTM have achieved high accuracy in violence detection.

However, these models are computationally intensive and may overfit due to limited

datasets. Additionally, optical flow-based methods have been used to detect panic

situations, but they suffer from high computational complexity. Snatching, such as theft

or pickpocketing, is a criminal behavior characterized by sudden movements and rare

interactions. Universal Attribute Models (UAM) based on GMM have been used to

extract high-dimensional action features for snatching detection. However, these models

often misclassify regular interactions as snatching due to the lack of limb motion

tracking. CNN-based models, such as VGG-19, have been employed to detect snatching

behavior by analyzing body motion and facial expressions. While these models perform

well in uncrowded scenarios, they struggle with group activities and require significant

computational resources. Sexual abuse detection, particularly in the context of

cyberbullying and child exploitation, is a challenging task. Fuzzy logic-based

models have been used to recognize offensive content in text messages, but they fail to

handle image-based harassment. CNN-based frameworks, such

as MobileNet and ResNet, have been employed to detect criminal behaviors by analyzing

facial features and image appearances. These models perform well in identifying age

groups and pornographic content but struggle with long-distance and low-resolution

images15.

The design and implementation of the system for real-time abnormal behavior

detection in smart surveillance systems are guided by two key principles: (1) the solution

must support real-time detection on modern hardware, and (2) it should be compatible

with multiple camera types, including wide-angle lenses with higher fields of view

(FoV). To achieve these goals, three distinct approaches were implemented: (1) a Stacked

Bidirectional LSTM Classifier, (2) a Spatiotemporal Autoencoder, and (3)

a Spatiotemporal LSTM Classifier. Each approach addresses different aspects of

abnormal behavior detection, leveraging pose estimation, spatiotemporal feature

extraction, and hybrid learning techniques. The first approach, Stacked Bidirectional

LSTM Classification via Pose Estimation, uses a Stacked Bidirectional LSTM

Classifier to detect abnormal behaviors based on pose estimation. The pipeline consists of

four stages: pose estimation, tracking, feature extraction, and classification. In the pose

estimation stage, the Regional Multi-person Pose Estimation (RMPE) framework with

a VGG19 backend is used to detect 18 key points on the human body. These key points

are refined using parametric pose non-maximum suppression to improve accuracy. While

VGG19 provides high accuracy, it is computationally expensive, so lightweight models

like MobileNet v2 are used for real-time inference on lower-spec hardware. Data

augmentation is applied by introducing noise to the extracted key points, which improves

the model's generalization and accuracy. In the tracking stage, cross-frame poses are
matched to form pose flows. A distance matrix is used to associate poses across frames,

with heuristics such as joint positions, average height, and height differences between

frames. Skeletons are sorted based on their distance from the image center, and matching

is performed using the distance between joints. Occlusions and missing joints are

addressed by reconstructing joint positions based on their relative positions in previous

frames. A default "idle" pose (e.g., a sitting position) is used as a fallback when joint data

is missing. The detected and tracked key points are then converted into feature vectors for

classification. Features include joint positions, average skeleton height, joint velocity,

and body velocity. A circular buffer stores skeleton data for a fixed number of frames

(e.g., 5 frames), and feature extraction occurs when the buffer is full. The buffer size is

optimized to balance accuracy and performance, with larger buffers leading to lower

accuracy in crowded scenarios. The classification model consists of Bidirectional LSTM

layers with three hidden layers of size 32x64 and ReLU activation. The model is trained

end-to-end and achieves 99.6% accuracy after 300 epochs. The classification process

takes less than 50 ms per frame, making it suitable for real-time applications. This

approach demonstrates the effectiveness of using pose estimation and LSTM-based

classification for detecting abnormal behaviors in real-time. The second

approach, Spatiotemporal Autoencoder, uses a Spatiotemporal Autoencoder to detect

abnormal events based on the dissimilarity between input and reconstructed frames. The

model is trained on normal events, and abnormal events are detected by thresholding

the reconstruction error. The autoencoder consists of an encoder and a decoder. The

encoder reduces the dimensionality of the input frames, while the decoder reconstructs

the frames. The model uses non-linear activation functions to extract useful features,
outperforming linear methods like PCA. Convolutional layers extract spatial features

from the input frames, with the number of filters carefully balanced to avoid performance

degradation while ensuring sufficient feature extraction. Frames are preprocessed by

resizing to 227x227, normalizing pixel values, and converting to grayscale. Techniques

like optical flow and frame subtraction are used to enhance temporal features. Data

augmentation is applied in the temporal dimension using strides to improve

generalization across variable frame rates. The spatiotemporal autoencoder

combines spatial and temporal feature extractors. The spatial encoder-decoder uses

convolutional layers, while the temporal encoder uses ConvLSTM layers to model

temporal dependencies. The model learns regular patterns in the training data, enabling it

to detect anomalies based on reconstruction error. The reconstruction error is calculated

as the Euclidean distance between input and reconstructed frames. An irregularity score is

derived by scaling the error between 0 and 1, and the regularity score is obtained by

subtracting the irregularity score from 1. A low regularity score indicates an abnormal

event. A variable thresholding technique is used to determine abnormal events in real-

time. During a "warm-up" session, the average reconstruction error is calculated to set an

adaptive threshold. This approach avoids the pitfalls of fixed thresholds and improves

detection sensitivity. The spatiotemporal autoencoder approach provides a robust method

for detecting anomalies by leveraging the dissimilarity between normal and abnormal

events. The third approach, Hybrid LSTM Classification, combines the Spatiotemporal

Autoencoder with an LSTM Classifier to reduce false positives and improve detection

accuracy. This hybrid model leverages the strengths of both unsupervised and supervised

learning. The hybrid model consists of the autoencoder encoder and a stacked LSTM
Classifier. The autoencoder acts as a high recaller, detecting most anomalies (including

false positives), while the LSTM Classifier refines the results to improve [Link]

training process involves three stages: (1) Unsupervised Training, where the autoencoder

is trained to learn regular patterns; (2) Transfer Learning, where the encoder weights are

transferred to the hybrid model, with the encoder layers marked as non-trainable; and

(3) Supervised Training, where the LSTM Classifier is trained on labeled data to

distinguish between true and false positives. The hybrid model achieves 98%

accuracy after 20 epochs, demonstrating its effectiveness in reducing false positives while

maintaining high recall. This approach highlights the benefits of combining unsupervised

and supervised learning techniques for robust anomaly detection. The three approaches—

Stacked Bidirectional LSTM Classification, Spatiotemporal Autoencoder, and Hybrid

LSTM Classification—provide a comprehensive framework for real-time abnormal

behavior detection in smart surveillance systems. Each approach addresses different

challenges, such as pose estimation, spatiotemporal feature extraction, and false positive

reduction. The system is designed to operate on modern hardware and support multiple

camera types, making it versatile and scalable for various applications. Future work could

focus on further optimizing computational efficiency and exploring unsupervised learning

techniques to reduce reliance on labeled data. These advancements will enhance the

system's ability to detect and respond to abnormal behaviors in real-time, improving

security and safety in various environments16.

The frameworks described in were tailored to address specific challenges in anomaly

detection, activity recognition, and surveillance, leveraging advanced machine learning

and deep learning techniques. Each system has unique objectives, methodologies, and
components, depending on its application domain. For instance, Smart Guard is

designed for accurate user behavior sequence anomaly detection in smart homes,

focusing on improving reconstruction accuracy and detecting anomalies in real-time. It

employs a Loss-guided Dynamic Mask Strategy, a Three-level Time-aware Positional

Encoder, and an attention-based encoder-decoder architecture to achieve its goals5. In

contrast, foreground object detection systems aim to identify suspicious activities by

detecting moving objects and differentiating them from static backgrounds. These

systems rely on background subtraction techniques, object tracking, and feature

extraction to achieve their objectives8.

Supervised and semi-supervised methods, on the other hand, aim to model normal and

abnormal behaviors using labeled data. These methods are particularly effective for

detecting specific predefined abnormal behaviors, such as fighting, loitering, or falling.

Techniques like the Bag of Words (BOW) approach and Support Vector Machine

(SVM) classifiers are commonly used in these systems. Transfer learning and

Convolutional Neural Networks (CNNs) are employed to improve performance on

related tasks by leveraging pre-trained models, especially in domains with limited labeled

data, such as medical imaging or natural language processing. These systems use

convolutional layers for feature extraction and pooling layers for dimensionality

reduction, making them highly effective for visual data analysis.

Intelligent video surveillance systems integrate hardware and software components to

automatically detect, analyze, and respond to events of interest in real-time. These

systems typically include CCTV cameras, preprocessing modules, motion detection,

feature extraction, and classification components. Human Activity Recognition (HAR)

systems focus on recognizing human activities from video data, with an emphasis on

classifying activities as normal or abnormal. These systems involve preprocessing,

motion detection, data filtration, feature extraction, and classification stages, often using

machine learning models like CNNs and Recurrent Neural Networks (RNNs).

The Abnormality Converging Scene Analysis Method (ACSAM) is designed to detect

abnormal behavior in crowded scenes using CNNs and textural features. It combines

image processing techniques with deep learning to extract textural and sequence-based

features, making it suitable for smart surveillance systems in crowded public spaces like

airports and stadiums.

Real-time abnormal behavior detection systems aim to detect abnormal behaviors in real-

time using various deep learning techniques, including Long Short-Term Memory

(LSTM) networks and autoencoders. These systems use pose estimation, spatiotemporal

feature extraction, and hybrid learning techniques to achieve their goals. For

example, Stacked Bidirectional LSTMs and Spatiotemporal Autoencoders are

employed to detect anomalies based on pose estimation and reconstruction errors. These

systems are designed to operate on modern hardware and support multiple camera types,

making them versatile and scalable for various applications9.

In summary, the architectural frameworks and system designs described in the text are

tailored to address specific challenges in anomaly detection, activity recognition, and

surveillance. While each system has its unique focus and methodology, they all leverage

advanced machine learning and deep learning techniques to improve accuracy and

efficiency. The choice of architecture and components depends on the specific

requirements and challenges of the application domain. Future advancements in these

systems will likely focus on optimizing computational efficiency, reducing reliance on

labeled data, and improving real-time performance to enhance their applicability in

various environments.

2.3 Related Works

Recent advancements in anomaly detection, activity recognition, and surveillance

systems have leveraged machine learning and deep learning techniques to address various

challenges. These works can be categorized based on their application domains,

methodologies, and objectives.

A context-aware system that uses supervised learning to recognize and monitor human

activities, particularly for detecting abnormal activities with minimal labeled data.

Designed for applications like smart home monitoring, especially for elderly individuals

living independently, the system employs multi-class SVM with kernel functions to

classify normal activities and detect anomalies, using a transition table to reduce

computational time. It integrates data from various sensors (e.g., infrared, physiological,

sound) for real-time activity recognition and incorporates human knowledge to describe

contexts, reducing the need for extensive sensor data collection. By transferring learned

knowledge across activities (e.g., Sit, Walk, Run), the system can generalize to new

activities without prior training. This non-invasive, low-cost, and flexible approach is

ideal for enhancing elderly care and smart home safety through real-time monitoring and

abnormal situation detection17.

The Smart Guard system focuses on accurate user behavior sequence anomaly detection

in smart homes. It employs a Loss-guided Dynamic Mask Strategy to prioritize hard-to-

learn behaviors and a Three-level Time-aware Positional Encoder to capture temporal

information. The system uses an attention-based encoder-decoder architecture for

sequence reconstruction and a Noise-aware Weighted Reconstruction Loss to detect

anomalies in real-time. This approach improves reconstruction accuracy and reduces

variance in reconstruction loss, making it suitable for ensuring the safety and security of

smart home environments5.

Foreground object detection is a critical step in identifying suspicious activities in

surveillance systems. Early approaches relied on single Gaussian models, which evolved

into multimodal Gaussian distributions like the Mixture of Gaussians (MOG). Techniques

such as background subtraction, object tracking, and feature extraction are commonly

used to detect moving objects. However, challenges like noise, shadows, and illumination

effects often lead to false classifications. Recent advancements have incorporated

techniques like color normalization, Gaussian smoothing, and fuzzy histogram color

normalization to improve [Link] methods for anomaly detection rely on

labeled data to model normal and abnormal behaviors. The Bag of Words

(BOW) approach, combined with Support Vector Machine (SVM) classifiers, has been

widely used for event detection in videos. Semi-supervised methods, on the other hand,

require only normal data for training and can be categorized into rule-based and model-

based approaches. Rule-based methods, such as those using sparse coding, detect

anomalies by identifying deviations from normal patterns. Model-based methods build

representations of normal behaviors and classify deviations as anomalies [Link]

learning has emerged as a powerful technique for improving performance on related tasks

by leveraging pre-trained models. This approach is particularly useful in domains with

limited labeled data, such as medical imaging and natural language processing. CNNs,

with their ability to automatically extract features from visual data, have been widely

adopted for tasks like image classification, object detection, and key point detection. For

video data, 3D CNNs have been developed to capture temporal dimensions, making them

suitable for action recognition and video classification [Link] systems have significantly

benefited from deep learning techniques, particularly CNNs and Recurrent Neural

Networks (RNNs). CNNs are well-suited for recognizing complex human gestures and

activities, automating feature extraction, and handling nonlinear relationships in time-

domain data. RNNs, especially Long Short-Term Memory (LSTM) networks, are

effective for capturing temporal dependencies in sequential data. Hybrid architectures

combining CNNs and LSTMs have been proposed to improve real-time performance and

accuracy in HAR [Link] Abnormality Converging Scene Analysis Method

(ACSAM) is designed to detect abnormal behavior in crowded scenes using CNNs and

textural features. This method integrates image processing techniques with deep learning

to extract spatial and temporal features, enabling real-time anomaly detection. ACSAM

categorizes scenes into individual and crowd behaviors and differentiates between normal

and abnormal activities using specialized training layers. However, its performance is

limited by computational resource requirements and the need for large labeled

[Link]-time abnormal behavior detection systems leverage pose estimation,

spatiotemporal feature extraction, and hybrid learning techniques. Approaches

like Stacked Bidirectional LSTMs and Spatiotemporal Autoencoders have been

developed to detect anomalies based on pose estimation and reconstruction errors. These

systems are designed to operate on modern hardware and support multiple camera types,
 making them versatile for various applications. Hybrid models combining unsupervised

and supervised learning techniques have been proposed to reduce false positives and

improve detection accuracy16. Some other related works are:

S/ Authors Year of Title of the Methods used in the study Limitations of the study

N the study

study

1 Adeleh Farzad 2015 Real-time Smart  Hidden Markov Model  Dataset Size and

and Rahebeh Situation (HMM) Diversity

Niaraki Asli Awareness based  Leave-one-out method  Generalization to

on Camera Real-World

Array and Scenarios

Machine

Learning

2 Jiyoo Lee 1 , 2020 Real-time video  I3D model(inflated 3D  Model

Seung-Jung Shin Surveillance ConvNet) Generalization

System Design  3D CNN,  Real-Time

Proposal Using performance

Abnormal  False Positives and

Behavior Negatives

Recognition

Technology

3 Yubiao 2022 Research on  Multi-Index  Algorithm

 Wang, Junhao Abnormal Regression Dependence

Wen, Wei Behavior  Neural Factorization  Data Quality and

Zhou, Quanwang Prediction by Machine Completeness

Wu, Yingchun Integrating  Semantic Analysis

Wei, Heng Multiple Indexes

Li, Bamei Tao of Student

Behavior and

Text Information

in Big Data

Environment

4 Rajesh Sharma 2021 An Efficient  Gaussian Mixture  Background

& Akey Dimension Models (GMM) Subtraction

Sungheetha Reduction based  Convolutional Neural Challenges

Fusion of CNN Networks (CNN)  Threshold

and SVM Model  Support Vector Sensitivity

for Detection of Machines (SVM)  Real time processing

Abnormal

Incident in

Video

Surveillance

5 Tejashri Subhash 2021 Methodology for  CNN-based supervised  Threshold

Bora1 , Monika human learning Sensitivity

Dhananjay suspicious  Generalization

Rokade activity

detection

2.4 Summary of gaps in literature

Despite their advancements, these systems face several challenges. Smart Guard must

handle noise behaviors and improve reconstruction accuracy to ensure reliable anomaly

detection. Foreground object detection systems struggle with challenges like noise,

shadows, and illumination effects, which can lead to false classifications. Supervised and

semi-supervised methods require large, labeled datasets and are sensitive to parameter

tuning, which can limit their scalability. Transfer learning and CNNs face challenges

related to computational complexity and the need for task-specific fine-tuning. Intelligent

video surveillance systems must address challenges like real-time processing,

computational efficiency, and handling diverse environments. HAR systems face

challenges such as class variety, real-time performance, and computational complexity.

ACSAM requires significant computational resources and may struggle with

generalization across different datasets. Real-time abnormal behavior detection systems

must manage challenges like computational efficiency, occlusions, and reducing false

positives.
For Smart Guard, future research could focus on further improving reconstruction

accuracy and handling more complex noise behaviors. Foreground object detection

systems could benefit from enhanced techniques for noise reduction and improved object

tracking accuracy. Supervised and semi-supervised methods could explore unsupervised

learning techniques to reduce reliance on labeled data. Transfer learning and CNNs could

focus on optimizing computational efficiency and exploring new architectures for better

feature extraction. Intelligent video surveillance systems could integrate additional data

sources and improve real-time processing capabilities. HAR systems could adopt hybrid

approaches combining CNNs, LSTMs, and Extreme Learning Machines (ELMs) for

improved performance. ACSAM could focus on optimizing computational complexity

and exploring unsupervised learning techniques. Real-time abnormal behavior detection

systems could further optimize computational efficiency and explore unsupervised

learning techniques to reduce reliance on labeled data.

END NOTES
1. Esam A. AlQaralleh, Fahad Aldhaban, and Bassam A. Y. Alqaralleh, "Smart Deep

Learning-Based Human Behavior Classification for Video Surveillance," Computers,

Materials & Applications, 2022.

2. Stambolian, D. Zh., et al. "Development of an Algorithm for Abnormal Human

Behavior Detection in Intelligent Video Surveillance System." IOP Conference

Series: Materials Science and Engineering 1069 (2021): 012046.

[Link]
3. Mu, H., Sun, R., Yuan, G., and Wang, Y. "Abnormal Human Behavior Detection in

Videos: A Review." Information Technology and Control 50, no. 3 (2021): 522-545.

[Link]

4. Rafsanjani, Mohammad Sadat Hussain, and Ahmedul Kabir. "Violent Human

Behavior Detection from Videos Using Machine Learning." Institute of Information

Technology, University of Dhaka.

5. Xiao, Jingyu, Zhiyao Xu, Qingsong Zou, Qing Li, Dan Zhao, Dong Fang, Ruoyu Li,

Wenxin Tang, Kang Li, Xudong Zuo, Penghui Hu, Yong Jiang, Zixuan Weng, and

Michael R. Lyu. "Make Your Home Safe: Time-Aware Unsupervised User Behavior

Anomaly Detection in Smart Homes via Loss-Guided Mask." arXiv:2406.10928v2

[[Link]] 18 Jun 2024.

6. Sedaghati, N., Ardebili, S., and Ghaffari, A. "Application of Human Activity/Action

Recognition: A Review." Multimedia Tools and Applications, 2025.

[Link]

7. Deng, L., Fu, R., Sun, Q., Jiang, M., Li, Z., Chen, H., Yu, Z., & Bu, X. (2022).

“Abnormal behavior recognition based on feature fusion C3D network”. Journal of

Electronic Imaging, 32, 021605 - 021605.

8. Manisha Mudgal, Deepika Punj, and Anuradha Pillai, "Suspicious Action Detection in

Intelligent Surveillance System Using Action Attribute Modelling," Department of

Computer Engineering, JC Bose UST YMCA Faridabad, Haryana, India, published

February17,2021.

[Link]
9. Mabrouk, Amira Ben, and Ezzeddine Zagrouba. "Abnormal Behavior Recognition for

Intelligent Video Surveillance Systems." Expert Systems with Applications: An

International Journal 91 (2017): 480-491.

[Link]

10. Pavitra, T., and Rajasekaran Thangaraj. "Real Time Abnormal Behavior Detection

and Warning System Based on Deep Convolutional Neural Network." Nandha

Engineering College, Erode, Tamil Nadu, India.

[Link]

11. Pooja Br, and Rajkumar N. "Real-Time Intelligent Video Surveillance System Using

Recurrent Neural Network." Procedia Computer Science 235 (2024): 1522-1531.

[Link]

12. Chenglizhao Chen, Xinyu Liu, Mengke Song, Luming Li, Xu Yu, and Shanchen

Pang, "Unveiling Context-Related Anomalies: Knowledge Graph Empowered

Decoupling of Scene and Action for Human-Related Video Anomaly Detection,"

College of Computer Science and Technology, China University of Petroleum (East

China).

13. Yao, Fenghui and Guifeng Shao. “Real-time Smart Situation Awareness based on

Camera Array and Machine Learning.” The Proceedings of the 9th IIAE

International Conference on Industrial Application Engineering 2020 (2021).

[Link]

14. Wu, Yangkai, Luhua Qiu, Jinming Wang, and Shujuan Feng. “The Use of

Convolutional Neural Networks for Abnormal Behavior Recognition in Crowd

 Scenes.” Information Processing & Management 62, no. 1 (2025): 103880.

[Link]

15. Kuppusamy, P., and V. C. Bharathi. “Human Abnormal Behavior Detection Using

CNNs in Crowded and Uncrowded Surveillance – A Survey.” Measurement: Sensors

24 (2022): 100510. [Link]

16. Tsiktsiris, Dimitris, Nikolaos Dimitriou, Antonios Lalas, Minas Dasygenis,

Konstantinos Votis, and Dimitrios Tzovaras. 2020. "Real-Time Abnormal Event

Detection for Enhanced Security in Autonomous Shuttles Mobility

Infrastructures" Sensors 20, no. 17: 4943. [Link]

17. Deshmukh, Kshitija, Aishwarya Kokane, Shweta Konde, and Guide-Gaur.

“Abnormal Activity Detection from Video Using SVM Algorithm.” International

Research Journal of Engineering and Technology (IRJET) 7, no. 6 (June 2020):

3875. [Link]

18.