Great!
Here's a comprehensive, detailed set of notes for all 5 units of Data Analytics (KCS-051)
based strictly on your syllabus. This content will help you understand concepts deeply and revise
thoroughly before your exam.
UNIT I: Introduction to Data Analytics & Lifecycle
1. Introduction to Data Analytics
Definition:
Data analytics refers to the process of examining raw data to draw conclusions, discover patterns,
and support decision-making.
2. Sources and Nature of Data
• Structured Data: Stored in relational databases (e.g., rows and columns in SQL).
• Semi-structured Data: Does not conform to relational databases but has tags or markers
(e.g., XML, JSON).
• Unstructured Data: No predefined format (e.g., images, audio, video, text documents).
3. Characteristics of Big Data
• Volume: Vast amounts of data (terabytes to zettabytes).
• Velocity: Speed at which data is generated and processed.
• Variety: Different data types (text, images, video, etc.).
• Veracity: Uncertainty or trustworthiness of data.
• Value: Usefulness of data in generating insights.
4. Big Data Platform
• A computing platform that handles large-scale data processing, typically using distributed
storage and parallel processing (e.g., Hadoop, Spark).
5. Evolution of Analytic Scalability
• From manual analysis to automation using AI/ML.
• Scalable from small datasets to big data environments.
6. Data Analytics Lifecycle
Phases:
1. Discovery: Identify business goals, understand resources.
2. Data Preparation: Clean, normalize, and transform data.
3. Model Planning: Choose analytical techniques (e.g., clustering, regression).
4. Model Building: Use tools like R, Python to build models.
5. Communicate Results: Use dashboards, reports, visualizations.
6. Operationalize: Deploy the model for real-time or batch processing.
UNIT II: Data Analysis Techniques
1. Regression Modeling
• Linear Regression: Relationship between dependent and independent variables.
• Multivariate Regression: Multiple independent variables.
• Logistic Regression: For binary classification.
2. Multivariate Analysis
• Analysis involving multiple variables to understand relationships and effects.
• Tools: MANOVA, PCA.
3. Bayesian Modeling
• Uses Bayes’ Theorem for prediction.
• Bayesian Networks: Graph-based representation of probabilistic relationships.
4. Support Vector Machines (SVM)
• Supervised ML algorithm for classification/regression.
• Works by finding a hyperplane that separates data.
5. Kernel Methods
• Transform data into higher dimensions to make it linearly separable.
• Used with SVM.
6. Time Series Analysis
• Analyze data over time (e.g., stock prices).
• Linear Systems Analysis: AR, MA, ARIMA models.
• Nonlinear Dynamics: Handles chaotic systems.
• Rule Induction: Derives logical rules from data.
7. Neural Networks
• Artificial networks that simulate brain learning using layers of nodes.
• Applications: Image recognition, NLP, classification.
8. Principal Component Analysis (PCA)
• Dimensionality reduction technique.
• Transforms correlated variables into fewer uncorrelated ones (principal components).
9. Fuzzy Decision Trees
• Use fuzzy logic to handle uncertainty in classification.
10. Stochastic Search
• Random sampling for optimization when traditional methods fail.
UNIT III: Mining Data Streams
1. Stream Concepts
• Deals with continuous, high-speed data.
• Examples: Sensor data, real-time logs.
2. Stream Data Model and Architecture
• Must support:
o High throughput
o Low latency
o Fault tolerance
3. Stream Computing
• Batch vs. Stream:
o Batch: Static datasets
o Stream: Real-time processing
4. Sampling and Filtering
• Sampling: Process a subset of stream data.
• Filtering: Remove unwanted data using filters.
5. Counting Distinct Elements
• Bloom Filter: Memory-efficient method.
• HyperLogLog: Probabilistic counting algorithm.
6. Sliding and Decaying Windows
• Sliding Window: Fixed-size recent data is maintained.
• Decaying Window: Recent data has higher weight.
7. Estimating Moments
• Used in frequency counting or approximation.
• Track trends or summarize data.
8. Counting Oneness
• Measures uniqueness in a stream.
9. Real-Time Analytics Platform (RTAP)
• Tools: Apache Kafka, Apache Flink, Apache Storm.
• Use case: Fraud detection, recommendation engines.
10. Case Studies
• Stock Market Predictions: Real-time updates.
• Sensor Data Streams: IoT applications.
UNIT IV: Frequent Itemsets & Clustering
1. Mining Frequent Itemsets
• Market Basket Analysis: Find associations between items.
• Apriori Algorithm: Uses minimum support and confidence.
o Iterative: Generates k-itemsets from (k-1)-itemsets.
2. Challenges:
• High computation in large datasets
• Reducing redundancy
3. Handling Large Data in Memory
• Limited Pass Algorithm: Only few passes over data.
• FP-Growth: More efficient than Apriori.
4. Clustering Techniques:
A. Hierarchical Clustering:
• Builds a tree (dendrogram).
• Agglomerative (bottom-up) or Divisive (top-down)
B. K-means Clustering:
• Partitions data into k clusters using centroids.
• Iterative method to minimize intra-cluster variance.
C. High-Dimensional Clustering:
• Curse of dimensionality
• Use of CLIQUE, ProCLUS:
o CLIQUE: Combines density and grid-based clustering.
o ProCLUS: Subspace clustering using projected dimensions.
D. Non-Euclidean Clustering:
• Uses similarity measures other than Euclidean (e.g., cosine similarity).
UNIT V: Frameworks, Visualization & R
1. Big Data Frameworks
A. MapReduce:
• Programming model for parallel computation.
• Two phases: Map (filtering, sorting), Reduce (aggregation).
B. Hadoop Ecosystem:
• HDFS: Distributed storage.
• YARN: Resource manager.
• Hive: SQL-like querying.
• Pig: Data flow language.
• HBase: Columnar NoSQL database.
2. NoSQL Databases
• Non-relational, scalable databases.
• Types: Document-based (MongoDB), Columnar (HBase), Graph (Neo4j).
3. Distributed File Systems
• Store data across multiple nodes.
• Hadoop Distributed File System (HDFS)
4. Visualization Techniques
• Tools: Tableau, Power BI, matplotlib, ggplot2
• Descriptive Statistics: Mean, median, variance.
• Exploratory Data Analysis (EDA): Graphs, plots, distributions.
5. R Programming
• Open-source statistical tool.
• Import/Export: [Link](), [Link]()
• Data Types: Vectors, matrices, data frames.
• Summary Stats: summary(), mean(), sd()
• Plotting: plot(), hist(), boxplot()
Sample Code:
data <- [Link]("[Link]")
summary(data)
plot(data$marks, data$attendance)
Exam Tips:
• Focus on Apriori, PCA, K-means, MapReduce, and Data Lifecycle.
• Understand how stream data differs from traditional batch processing.
• Be ready to write short R scripts for visualization or importing data.
• Practice case studies and examples from real-world analytics.