EX.

NO: 1 Classification of Iris Flowers using Pandas

DATE:
& Decision Tree

AIM:
To Write a Program Classification of Iris Flowers using Pandas & Decision
Tree.

ALGORITHM:
Step 1: Load dataset from [Link].
Step 2: Convert to DataFrame for easy inspection and add a human-readable
label column.
Step 3: Define features X and target y.
Step 4: (Optional) Sanity checks: check for missing values, class balance, basic
stats / histograms.
Step 5: Split data into X_train, X_test, y_train, y_test.
Step 6: Create a DecisionTreeClassifier with chosen hyperparameters (e.g.,
random_state, max_depth).
Step 7: Train (fit) the model on training data.
Step 8: Predict labels for the test set.
Step 9: Evaluate using accuracy_score (and optionally confusion matrix,
classification report).
Step 10: Visualize the trained tree with plot_tree() (or export as graph).
Step 11: (Optional) Save the model to disk.
Step 12: Tune hyperparameters via cross-validation
GridSearchCV if performance is unsatisfactory.

CODE:
#Install the required libraries
Make sure these libraries are installed in your system :
pip install pandas scikit-learn matplotlib
#Import the libraries
import pandas as pd
from [Link] import load_iris
from sklearn.model_selection import train_test_split
from [Link] import DecisionTreeClassifier, plot_tree
from [Link] import accuracy_score
import [Link] as plt

1. Load the Iris dataset

iris = load_iris()
# Convert to a pandas DataFrame for easy analysis
df = [Link](data=[Link], columns=iris.feature_names)
df['species'] = [Link]

# Map target numbers to flower names

df['species'] = df['species'].map({0: 'setosa', 1: 'versicolor', 2: 'virginica'})
print([Link]())

2. Split the dataset into training and testing sets

X = [Link][:, :-1] # features (sepal length, width, etc.)
y = [Link][:, -1] # target (species)
# 80% training and 20% testing
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)

3. Train the Decision Tree Classifier

model = DecisionTreeClassifier(random_state=42)
[Link](X_train, y_train)

4. Make predictions and check accuracy

 y_pred = [Link](X_test)
accuracy = accuracy_score(y_test, y_pred)
print("Decision Tree Accuracy:", accuracy)

5. Visualize the Decision Tree

[Link](figsize=(12,8))
plot_tree(model, feature_names=iris.feature_names,
class_names=iris.target_names, filled=True)
[Link]()

OUTPUT:
RESULT:

Thus code has been executed and verified successfully.

[Link] Classification using Logistic Regression
DATE:

AIM:
To Write a Classification using Logistic Regression.

ALGORITHM:
Step 1: Import libraries — load the functions you need (dataset, split, model,
metrics).
Step 2: Load dataset — read Iris features X and labels y.
Step 3: Split — divide into training (X_train, y_train) and testing (X_test,
y_test) sets.
Step 4: Create model — instantiate LogisticRegression(max_iter=200) (sets
solver + convergence limit).
Step 5: Train model — [Link](X_train, y_train) learns model parameters
from training data.
Step 6: Predict & evaluate — [Link](X_test) and compute accuracy,
confusion_matrix, classification_report.

CODE:

#Import required libraries

from [Link] import load_iris

from sklearn.model_selection import train_test_split

from sklearn.linear_model import LogisticRegression

from [Link] import accuracy_score, classification_report,

confusion_matrix

Step 1: Load dataset

 iris = load_iris()

X = [Link]

y = [Link]

Step 2: Split data into training and testing sets (80% train, 20% test)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,

random_state=42)

Step 3: Create a Logistic Regression model

model = LogisticRegression(max_iter=200)

Step 4: Train the model

[Link](X_train, y_train)

Step 5: Make predictions

y_pred = [Link](X_test)

Step 6: Evaluate model performance

print("Accuracy:", accuracy_score(y_test, y_pred))

print("\nConfusion Matrix:\n", confusion_matrix(y_test, y_pred))

print("\nClassification Report:\n", classification_report(y_test, y_pred))

OUTPUT:
RESULT:

Thus code has been executed and verified successfully.

[Link]:3D LDA with ML Pipeline (Advanced)
DATE:

AIM:

To Write program LDA with ML Pipeline (Advanced).

ALGORITHM:
Step 1 : Load Iris dataset (load_iris) → provides features (X) and labels (y).

Step 2 : Split data (train_test_split) → divides into training/testing sets.

Step 3 : Scale features (StandardScaler) → normalize data for better

convergence.

Step 4 : Apply dimensionality reduction (LinearDiscriminantAnalysis) →

reduce feature space while maximizing class separability.

Step 5 : Train classifier (LogisticRegression) → predicts labels.

Step 6 : Evaluate model (accuracy_score, confusion_matrix,

classification_report) → measures performance.

CODE:

#Import required libraries

from [Link] import load_iris

from sklearn.model_selection import train_test_split

from [Link] import StandardScaler

from sklearn.discriminant_analysis import LinearDiscriminantAnalysis

from sklearn.linear_model import LogisticRegression

from [Link] import Pipeline

from [Link] import accuracy_score, classification_report,

confusion_matrix

Step 1: Load the dataset

iris = load_iris()

X = [Link] # Features

y = [Link] # Labels

Step 2: Split the data into training and testing sets (80% train, 20% test)

X_train, X_test, y_train, y_test = train_test_split(

X, y, test_size=0.2, random_state=42, stratify=y

Step 3: Create a Machine Learning Pipeline

pipeline = Pipeline([

('scaler', StandardScaler()),

('lda', LinearDiscriminantAnalysis(n_components=2)),

('classifier', LogisticRegression(max_iter=200))

])

Step 4 : Train the pipeline

[Link](X_train, y_train)

Step 5: Make predictions

y_pred = [Link](X_test)

Step 6: Evaluate the model

 print("Accuracy:", accuracy_score(y_test, y_pred))

print("\nConfusion Matrix:\n", confusion_matrix(y_test, y_pred))

print("\nClassification Report:\n", classification_report(y_test, y_pred))

OUTPUT:
RESULT:

Thus code has been executed and verified successfully.

 [Link] PCA + Clustering using KMeans
DATE:

AIM:
To Write a Program to PCA + Clustering using KMeans.

ALGORITHM:
Step 1: Start the process.
Step 2: Open your Python IDE or code editor (e.g., VS Code, PyCharm, Jupyter
Notebook).
Step 3: Create a new Python file (e.g., pca_kmeans.py).
Step 4: Import the required libraries:
[Link] → for plotting clusters.
load_iris → to load the Iris dataset.
StandardScaler → to scale features.
PCA → to reduce dimensions.
KMeans → for clustering.
silhouette_score and confusion_matrix → to evaluate clustering performance.
Step 5: Load the dataset:
Load Iris dataset into iris.
Assign X = [Link] (features) and y = [Link] (true labels, optional).

Step 6: Scale the features:

Create a StandardScaler object.
Fit and transform X to X_scaled.
Scaling ensures all features contribute equally to PCA and
KMeans.
Step 7: Apply PCA for dimensionality reduction:
 Create a PCA object with n_components=2.
Fit and transform X_scaled to X_pca.
PCA reduces dimensions for easier visualization and clustering.
Step 8: Apply KMeans clustering:
Create a KMeans object with n_clusters=3 and random_state=42.
Fit KMeans on X_pca.
Assign cluster labels to clusters = kmeans.labels_.
Step 9: Evaluate the clustering:
Calculate silhouette score for X_pca and clusters to measure cluster quality.
Print the silhouette score.
Optionally, compare clusters with true labels using a confusion matrix.
Step 10: Visualize the clusters:
Create a figure using [Link]().
Plot X_pca points using [Link](), colored by cluster assignments.
Overlay cluster centroids with red X markers.
Add labels for axes (PCA Component 1 and PCA Component 2).
Add a title and legend.
Display the plot using [Link]().
Step 11: End the process.

CODE:
Import required libraries
import [Link] as plt
from [Link] import load_iris
from [Link] import StandardScaler
from [Link] import PCA
from [Link] import KMeans
from [Link] import silhouette_score, confusion_matrix

Step 1: Load dataset

iris = load_iris()
X = [Link]
y = [Link]

Step 2: Scale features

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

Step 3: Apply PCA to reduce dimensions to 2 (for visualization)

pca = PCA(n_components=2)
X_pca = pca.fit_transform(X_scaled)

Step 4: Apply KMeans clustering

kmeans = KMeans(n_clusters=3, random_state=42)
[Link](X_pca)
clusters = kmeans.labels_

Step 5: Evaluate clustering

sil_score = silhouette_score(X_pca, clusters)
print("Silhouette Score:", sil_score)

Step 6: Visualize clusters

[Link](figsize=(8,6))
[Link](X_pca[:,0], X_pca[:,1], c=clusters, cmap='viridis', marker='o',
edgecolor='k', s=100)
[Link](kmeans.cluster_centers_[:,0], kmeans.cluster_centers_[:,1], c='red',
marker='X', s=200, label='Centroids')
[Link]('PCA Component 1')
[Link]('PCA Component 2')
[Link]('KMeans Clustering on PCA-reduced Data')
[Link]()
[Link]()
OUTPUT:

RESULT:

Thus code has been executed and verified successfully.

[Link] Linear Regression Pipeline using Spark MLlib
DATE:
on House Price Data

AIM:
To Write a Program to Linear Regression Pipeline using Spark MLlib on
House Price Data.

ALGORITHM:
Step 1: Start the process.
Step 2: Initialize a SparkSession using [Link]().
Step 3: Load the house price dataset using [Link]() with header=True
and inferSchema=True.
Step 4: Display the schema using printSchema() to understand data types.
Step 5: Define the target variable (price) and use all other columns as features.
Step 6: Create a VectorAssembler to combine multiple feature columns into
one vector column (features).
Step 7: Initialize a LinearRegression model specifying featuresCol and
labelCol.
Step 8: Build a Pipeline with two stages:
Stage 1: VectorAssembler (feature transformation)
Stage 2: LinearRegression (model training)
Step 9: Split the dataset into 80% training and 20% testing sets.
Step 10: Train the model using [Link](train_data).
Step 11: Generate predictions on the test dataset using
[Link](test_data).
Step 12: Evaluate the model using RegressionEvaluator with metrics such as
RMSE and R².
Step 13: Display predicted vs actual prices.
Step 14: Stop the Spark session to free resources.
 Step 15: End the process.

CODE:
#Import required libraries
from [Link] import SparkSession
from [Link] import VectorAssembler
from [Link] import LinearRegression
from [Link] import Pipeline
from [Link] import RegressionEvaluator
Step 1: Initialize Spark Session
spark = [Link] \
.appName("LinearRegressionPipeline_HousePrice") \
.getOrCreate()
Step 2: Load the House Price dataset (replace with your file path)
Example CSV columns: "area", "bedrooms", "bathrooms", "price"
data = [Link]("house_price.csv", header=True, inferSchema=True)
Step 3: Display dataset schema
[Link]()
[Link](5)
Step 4: Define feature columns and label column
feature_columns = [col for col in [Link] if col != 'price']
Step 5: Assemble features into a single vector
assembler = VectorAssembler(
inputCols=feature_columns,
outputCol="features"
)
Step 6: Define Linear Regression model
lr = LinearRegression(
featuresCol="features",
labelCol="price"
)
Step 7: Create a Pipeline (Assembler → Linear Regression)
pipeline = Pipeline(stages=[assembler, lr])
Step 8: Split data into training and testing sets
train_data, test_data = [Link]([0.8, 0.2], seed=42)
Step 9: Train the model using the pipeline
model = [Link](train_data)
Step 10: Make predictions on test data
predictions = [Link](test_data)
Step 11: Evaluate the model performance
evaluator = RegressionEvaluator(
labelCol="price",
predictionCol="prediction",
metricName="rmse"
)
rmse = [Link](predictions)
r2 = [Link](predictions, {[Link]: "r2"})
print(f"Root Mean Squared Error (RMSE): {rmse}")
print(f"R² Score: {r2}")
Step 12: Display sample predictions
[Link]("features", "price", "prediction").show(10)
Step 13: Stop the Spark Session
[Link]()
[Link] Logistic Regression Pipeline on Titanic Dataset
DATE:
using Spark Mllib

AIM:
To Write a Program Logistic Regression Pipeline on Titanic Dataset using
Spark Mllib.

ALGORITHM:
Step 1: Start the process.
Step 2: Initialize a Spark Session using [Link]().
Step 3: Load the Titanic dataset (CSV format) using [Link]() with
header=True and inferSchema=True.
Step 4: Display the dataset schema and first few rows to confirm loading.
Step 5: Drop any rows with missing values in key columns like Age, Fare, Sex,
and Embarked.
Step 6: Convert categorical columns (Sex, Embarked) into numerical indices
using StringIndexer.
Step 7: Assemble all numeric and indexed categorical features into a single
vector using VectorAssembler.
Step 8: Initialize a LogisticRegression model with featuresCol="features" and
labelCol="Survived".
Step 9: Create a Pipeline containing:
StringIndexer for categorical variables.
VectorAssembler for feature vector creation.
LogisticRegression for classification.
Step 10: Split the data into training and testing subsets (80% training, 20%
testing).
Step 11: Train the pipeline model using [Link](train_data).
Step 12: Make predictions using [Link](test_data).
Step 13: Evaluate the model using BinaryClassificationEvaluator (AUC) and
 MulticlassClassificationEvaluator (Accuracy).
Step 14: Display model predictions along with survival probabilities.
Step 15: Stop the Spark session to free resources.
Step 16: End the process.

CODE:
# Import required libraries
from [Link] import SparkSession
from [Link] import StringIndexer, VectorAssembler
from [Link] import LogisticRegression
from [Link] import Pipeline
from [Link] import MulticlassClassificationEvaluator,
BinaryClassificationEvaluator

Step 1: Initialize Spark Session

spark = [Link] \
.appName("Titanic_LogisticRegression_Pipeline") \
.getOrCreate()

Step 2: Load the Titanic dataset (replace path as needed)

# Dataset should contain columns like: 'Survived', 'Pclass', 'Sex', 'Age',
'Fare', 'Embarked'
data = [Link]("[Link]", header=True, inferSchema=True)

Step 3: Display dataset schema and first few rows

[Link]()
[Link](5)

Step 4: Data Preprocessing

# Drop rows with missing values in important columns
data = [Link](subset=["Survived", "Pclass", "Sex", "Age", "Fare",
"Embarked"])

Step 5: Convert categorical columns into numeric using StringIndexer

sex_indexer = StringIndexer(inputCol="Sex", outputCol="SexIndex")
embarked_indexer = StringIndexer(inputCol="Embarked",
outputCol="EmbarkedIndex")

Step 6: Combine all feature columns into a single feature vector

assembler = VectorAssembler(
inputCols=["Pclass", "SexIndex", "Age", "Fare", "EmbarkedIndex"],
outputCol="features"
)

Step 7: Initialize Logistic Regression model

lr = LogisticRegression(featuresCol="features", labelCol="Survived")

Step 8: Create a Machine Learning Pipeline

pipeline = Pipeline(stages=[sex_indexer, embarked_indexer, assembler, lr])

Step 9: Split data into training (80%) and testing (20%) sets
train_data, test_data = [Link]([0.8, 0.2], seed=42)

Step 10: Train the Logistic Regression model

model = [Link](train_data)

Step 11: Make predictions on the test data

predictions = [Link](test_data)

Step 12: Evaluate model performance

# Binary classification evaluator (since Survived = 0 or 1)
 evaluator = BinaryClassificationEvaluator(labelCol="Survived",
rawPredictionCol="rawPrediction")

accuracy_evaluator = MulticlassClassificationEvaluator(
labelCol="Survived", predictionCol="prediction", metricName="accuracy"
)

accuracy = accuracy_evaluator.evaluate(predictions)
auc = [Link](predictions)

print(f"Accuracy: {accuracy:.4f}")
print(f"Area Under ROC (AUC): {auc:.4f}")

Step 13: Display sample predictions

[Link]("Survived", "prediction", "probability").show(10)

Step 14: Stop Spark session

[Link]()

OUTPUT:
[Link] Implement the following Data structures in Java
DATE: Linked Lists, Stacks, Queues, Set, Map

AIM:
To Write a Program to Implement the following Data structures in Java Linked
Lists, Stacks, Queues, Set, Map.

Algorithm Linked List:

Step 1: Start the process.
Step 2: Import the [Link] package to access LinkedList and Iterator classes.
Step 3: Define the class LinkedList1.
Step 4: Inside the class, define the main() method — this is the program’s entry point.
Step 5: Create a LinkedList object named al to store String elements.
→ Syntax: LinkedList<String> al = new LinkedList<String>();
Step 6: Add elements to the LinkedList using the add() method.
→ Add "Ravi", "Vijay", "Ravi", "Ajay".
Step 7: Create an Iterator object for the LinkedList.
→ Syntax: Iterator<String> itr = [Link]();
Step 8: Use a while loop to iterate through the LinkedList elements.
→ Condition: while ([Link]()) — checks if there are more elements.
Step 9: Inside the loop, print each element using [Link]([Link]());.
Step 10: Continue the loop until all elements are printed.
Step 11: Stop the process.

Code:
!sudo apt-get install openjdk-17-jdk -y
!java -version

Then create and run the Java file:

%%writefile [Link]
import [Link].*;
public class LinkedList1 {
public static void main(String args[]) {
LinkedList<String> al = new LinkedList<String>();
[Link]("Ravi");
[Link]("Vijay");
[Link]("Ravi");
[Link]("Ajay");
Iterator<String> itr = [Link]();
while ([Link]()) {
[Link]([Link]());
}
}
}

To Compile the Code:

!javac [Link]
!java LinkedList1

OUTPUT:
ALGORITHM FOR STACKS:
Step 1: Start the program.
Step 2: Import the required packages:
[Link].* and [Link].* to use Java’s built-in classes.
Step 3: Define the class StackDemo.
Step 4: Inside the main() method, create two Stack objects:
- stack1 (non-generic stack).
- stack2 (generic stack that stores String type).
Step 5: Push elements into stack1 using the push() method:
- Push "Nakka".
- Push "Sri Manohar".
- Push "Reddy".
Step 6: Display the elements of the stack using [Link](stack1).
Step 7: The stack elements are printed in the order they were pushed (LIFO structure).
Step 8: End the program.

Code :
%%writefile [Link]
import [Link].*;
import [Link].*;

class StackDemo {
public static void main(String[] args) {
// Create stacks
Stack stack1 = new Stack();
Stack<String> stack2 = new Stack<String>();

// Push elements into stack1

[Link]("Nakka");
 [Link]("Sri Manohar");
[Link]("Reddy");

// Display stack elements

[Link](stack1);
}
}

To Compile the Code:

!javac [Link]
!java StackDemo

OUTPUT:
Algorithm for Queues:

Step 1: Start the process.

Step 2: Import the required packages [Link] and [Link].
Step 3: Define the main class Main.
Step 4: Inside the main() method, create a Queue object named numbers using the
LinkedList class.
Step 5: Add (enqueue) elements into the queue using the offer() method.
• Insert 1 into the queue.
• Insert 2 into the queue.
• Insert 3 into the queue.
Step 6: Display the queue elements using [Link]().
Step 7: Remove (dequeue) the front element from the queue using the poll() method.
Step 8: Display the queue again to show the updated elements.
Step 9: End the process.

Code For Queues:

%%writefile [Link]

import [Link];

import [Link];

class Main {

public static void main(String[] args) {

Queue<Integer> numbers = new LinkedList<>();

[Link](1);

[Link](2);

[Link](3);

[Link]("Queue: " + numbers);

[Link]();

[Link]("Queue after dequeue: " + numbers);

 }

To Run the program:

!java Main

OUTPUT:

Algorithm for Set:

Step 1: Start the process.

Step 2: Import the required packages:
→ [Link]
→ [Link]

Step 3: Define the main class SetInterfaceExample.

Step 4: Inside the main() method, create a Set object named numbers using the
HashSet class.
Step 5: Add integer elements into the set using the add() method:
• Add 10 to the set.
 • Add 20 to the set.
• Add 30 to the set.
Step 6: Check if the set contains the element 20 using the contains() method.
• Print the result (true or false).
Step 7: Remove the element 30 from the set using the remove() method.
Step 8: Display the current contents of the set using [Link]().
Step 9: End the process.

Code for Set:

%%writefile [Link]

import [Link];

import [Link];

public class SetInterfaceExample {

public static void main(String[] args) {

// Create a HashSet

Set<Integer> numbers = new HashSet<>();

// Add elements

[Link](10);

[Link](20);

[Link](30);

// Check if 20 exists

[Link]("Contains 20: " + [Link](20));

 // Remove element 30

[Link](30);

// Display updated set

[Link]("Numbers Set: " + numbers);

OUTPUT:
Algorithm For Map:
Step 1: Start the process.

Step 2: Import the required packages:

→ [Link]
→ [Link]

Step 3: Define the main class MapInterfaceExample.

Step 4: Inside the main() method, create a Map object named countryCodes using the
TreeMap class.

Step 5: Add key-value pairs into the map using the put() method:
• Add key "USA" with value "+1".
• Add key "India" with value "+91".
• Add key "UAE" with value "+97".

Step 6: Check if the map contains the key "India" using the containsKey() method.
• Print the result (true or false).

Step 7: Remove the key "USA" from the map using the remove() method.

Step 8: Display the current contents of the map using [Link]().

Step 9: End the process.

Code for the Map:
%%writefile [Link]

import [Link];

import [Link];

public class MapInterfaceExample {

public static void main(String[] args) {

// Create a TreeMap

Map<String, String> countryCodes = new TreeMap<>();

// Add key-value pairs

[Link]("USA", "+1");

[Link]("India", "+91");

[Link]("UAE", "+97");

// Check if key "India" exists

[Link]("Contains key 'India': " +

[Link]("India"));

// Remove key "USA"

 [Link]("USA");

// Display updated map

[Link]("Country Codes: " + countryCodes);

To Run and Compile the Code:

!java MapInterfaceExample

OUTPUT:
RESULT:
Thus code has been executed and verified successfully.

[Link] (i) Perform setting up and Installing Hadoop in its

DATE:
three operating modes: Standalone,
Pseudo distributed, Fully distributed
AIM:
To write a Program to Perform setting up and Installing Hadoop in its three
operating modes: Standalone,Pseudo distributed, Fully distributed.
[Link] Mode:
In Standalone Mode, Hadoop operates as a single monolithic Java process, beneficial
for debugging and initial testing of MapReduce applications with small datasets.
Here's a revised breakdown of the setup process:
1. Download Hadoop Binary Package:
o Visit the Hadoop releases page and download the latest binary package.
o Extract the downloaded package to a directory (e.g., C:\hadoop-3.1.2).
2. Set Environment Variables:
o Set the following environment variables:
▪ HADOOP_HOME: Path to the Hadoop installation directory (e.g., C:\
hadoop-3.1.2).
▪ HADOOP_BIN: Path to the Hadoop binary directory (e.g., C:\hadoop-3.1.2\
bin).
▪ JAVA_HOME: Path to your JDK installation (ensure it’s version 1.8)
3. Edit PATH Environment Variable:
o Add %HADOOP_HOME%, %HADOOP_BIN%, and %JAVA_HOME%\
bin to your system’s PATH variable.
4. Create Folders for Datanode and Namenode:
o Inside the Hadoop installation directory (C:\hadoop-3.1.2), create an input
folder.
o Inside the input folder, create two subfolders: datanode and namenode.

5. Configure Hadoop:
o Edit the Hadoop configuration files (e.g., [Link], [Link], etc.) to
specify the paths and settings.
6. Run Hadoop Jobs:
o You can now run Hadoop jobs in Standalone Mode.
2. Pseudo-distributed Mode:
In Pseudo-Distributed Mode, Hadoop emulates a distributed environment on a single
machine, running each
daemon as a separate Java process. Hadoop software is installed on a Single Node,
where various daemons of
Hadoop run on the same machine as separate Java processes. These daemons include
NameNode, DataNode,
SecondaryNameNode, JobTracker, and TaskTracker, all operating within the confines
of a single server.
1. Repeat Steps 1-4 from Standalone Mode.
2. Configure Hadoop:
o Edit the Hadoop configuration files to set the appropriate values for the
pseudo-distributed environment.
o Specify localhost or [Link] as the hostname for all daemons.

3. Start Hadoop Services:

o Start the Hadoop daemons (Namenode, Datanode, ResourceManager,
NodeManager, etc.)using the appropriate scripts.

3. Fully Distributed Mode:

In Fully Distributed Mode, the daemons NameNode, JobTracker,
SecondaryNameNode (Optional and can be
runon a separate node) run on the Master Node. The daemons DataNode and
TaskTracker run on the Slave
Node. In Fully Distributed Mode, Hadoop runs on a cluster of machines. Each
machine hosts one or more
Hadoop daemons. Follow these steps:
1. Repeat Steps 1-4 from Standalone Mode.
2. Configure Hadoop:
o Edit the Hadoop configuration files to set the values specific to your cluster.
 o Specify the actual hostnames or IP addresses of the machines in the cluster.
3. Start Hadoop Services:
o Start the Hadoop daemons on each machine in the cluster.

4. Distribute Configuration Files:

o Copy the modified configuration files to all machines in the cluster.
5. Run Hadoop Jobs:
o Submit Hadoop jobs to the cluster.

# For [Link]
<property>
<name>[Link]</name>
<value>hdfs://localhost:9000</value>
</property>

# For [Link] or [Link]

<property>
<name>[Link]</name>
<value>1</value>
</property><property>
<name>[Link]</name>
<value>C:\hadoop\data\namenode</value>
</property><property>
<name>[Link]</name>
<value>C:\hadoop\data\datanode</value>
</property>

# For [Link]
<property>
<name>[Link]</name>
 <value>yarn</value>
</property>

# For [Link]
<property>
<name>[Link]-services</name>
<value>mapreduce_shuffle</value>
</property><property>
<name>[Link]</name>
<value>[Link]</value>
</property>

Setting up Java Environment for Hadoop:

Configuring Hadoop:
Fig. Core [Link]

Fig. [Link]
Fig. [Link]

Fig. [Link]
[Link] (ii)Use web based tools to monitor your Hadoop setup.
DATE:

Setting up Hadoop Environment:

Starting the Hadoop Server:
1. To start the Hadoop Servers, go to Command prompt and then run this command.
Syntax: [Link] (or) [Link] & [Link].
 [Link] Running

• After that all the demons will start running.

• Namenode
• Datanode
• Resourcemanager

• Nodemanager
After Successfully running all these demons we can access these in localhost ports.
This completes the installation part of Hadoop. [Link]
(ii)Use web-based tools to monitor your Hadoop setup.

Hadoop accesing through Web UI [Link]://localhost:8042/node

[Link]://localhost:8088/cluster [Link]://localhost:9870/[Link]#tab-overview
[Link]://localhost:9864/[Link]

Fig. Applications

Fig. NodeManager information

Fig. Namenode

Fig. DataNode
RESULT:
Thus code has been executed and verified successfully.
[Link] Install and Run Pig then write Pig Latin
DATE:
scripts to sort, group, join, project, and filter
your data.

AIM:
To Write a program install and run Apache Pig and write Pig Latin scripts to perform
sorting, grouping, joining, projecting, and filtering operations on a dataset.

ALGORITHM:
Step 1: Install Apache Pig and set up environment variables.
Step 2: Run Pig in local mode using the command:
pig -x local
Step 3: Create a sample input data file (e.g., [Link]) containing student details.
Step 4: Load the dataset into Pig using LOAD and PigStorage().
Step 5: Use FILTER to select records based on a condition.
Step 6: Use FOREACH ... GENERATE to project specific columns.
Step 7: Use GROUP to group records based on a specific field.
Step 8: Use ORDER to sort the data in ascending or descending order.
Step 9: Use JOIN to combine data from two relations.

Step 10: Display the output using DUMP command.

CODE:
[Link] Sample Data
Create a text file named [Link]:
1,John,CS,85
2,Mary,IT,92
3,Sam,CS,70
4,Anna,IT,88
5,Paul,CS,95

[Link] another file [Link]:

CS,Computer Science
IT,Information Technology

[Link] Pig (Local Mode)

pig -x local

[Link] Latin Script

You can save this as student_analysis.pig:
-- Load the student dataset
students = LOAD '[Link]' USING PigStorage(',')
AS (id:int, name:chararray, dept:chararray, marks:int);

-- Load the department dataset

departments = LOAD '[Link]' USING PigStorage(',')
 AS (code:chararray, dept_name:chararray);

-- FILTER: Select students who scored above 80

high_scorers = FILTER students BY marks > 80;

-- PROJECT: Select only name and marks

name_marks = FOREACH students GENERATE name, marks;

-- GROUP: Group students by department

grouped_by_dept = GROUP students BY dept;

-- SORT: Sort students by marks descending

sorted_students = ORDER students BY marks DESC;

-- JOIN: Join students with department details

joined_data = JOIN students BY dept, departments BY code;

-- Display outputs
DUMP high_scorers;
DUMP name_marks;
DUMP grouped_by_dept;
DUMP sorted_students;
DUMP joined_data;

[Link] the Script

pig -x local student_analysis.pig

[Link] Output
Filtered (marks > 80):
(2,Mary,IT,92)
(4,Anna,IT,88)
(5,Paul,CS,95)

Projected (name, marks):

(John,85)
(Mary,92)
(Sam,70)
(Anna,88)
(Paul,95)
Grouped by dept:
(CS,{(1,John,CS,85),(3,Sam,CS,70),(5,Paul,CS,95)})
(IT,{(2,Mary,IT,92),(4,Anna,IT,88)})

Sorted by marks descending:

(5,Paul,CS,95)
(2,Mary,IT,92)
(4,Anna,IT,88)
(1,John,CS,85)
(3,Sam,CS,70)

Joined with departments:

(1,John,CS,85,CS,Computer Science)
(3,Sam,CS,70,CS,Computer Science)
(5,Paul,CS,95,CS,Computer Science)
(2,Mary,IT,92,IT,Information Technology)
(4,Anna,IT,88,IT,Information Technology)

RESULT:
Thus code has been executed and verified successfully.
[Link] Install and Run Hive then use Hive to create,
DATE: alter, and drop databases, tables, views, functions,
and indexes.

AIM:
To install and run Apache Hive and use HiveQL to create, alter, and drop
databases, tables, views, functions, and indexes.

ALGORITHM:
Step 1: Install Apache Hive and configure it with Hadoop.
Step 2: Start the Hadoop and Hive services.
Step 3: Open the Hive command-line interface (CLI) using the command hive.
Step 4: Create a new Hive database using the CREATE DATABASE command.
Step 5: Create a table within the database using CREATE TABLE.
Step 6: Load data into the table using the LOAD DATA command.
Step 7: Perform queries to verify the data.
Step 8: Alter the table structure using the ALTER TABLE command.
Step 9: Create a VIEW from an existing table using CREATE VIEW.
Step 10: Create a FUNCTION for user-defined operations.
Step 11: Create an INDEX on a column for faster retrieval.
Step 12: Drop the created objects (database, table, view, index, etc.) using DROP
statements.

CODE:
[Link] and Run Hive

Make sure Hadoop is installed and running:

[Link]
[Link]

Download and set up Hive:

wget [Link]
tar -xvzf [Link]
sudo mv apache-hive-3.1.3-bin /usr/local/hive

Set environment variables in ~/.bashrc:

export HIVE_HOME=/usr/local/hive
export PATH=$PATH:$HIVE_HOME/bin

Reload and open Hive CLI:

source ~/.bashrc
hive

[Link] a Database
CREATE DATABASE college;
SHOW DATABASES;
USE college;

[Link] a Table
CREATE TABLE students (
id INT,
name STRING,
dept STRING,
marks INT
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE;

[Link] Data into the Table

Assume you have a file /home/hadoop/[Link]:

1,John,CS,85
2,Mary,IT,92
3,Sam,CS,70
4,Anna,IT,88
5,Paul,CS,95

LOAD DATA LOCAL INPATH '/home/hadoop/[Link]' INTO TABLE students;

SELECT * FROM students;

[Link] the Table

Add a new column:

ALTER TABLE students ADD COLUMNS (year INT);
DESCRIBE students;

Rename the table:

ALTER TABLE students RENAME TO student_info;
[Link] a View
CREATE VIEW high_scorers AS
SELECT name, dept, marks
FROM student_info
WHERE marks > 80;
SELECT * FROM high_scorers;

[Link] a Function
Hive includes built-in functions, but you can also create temporary ones.

Example: Use a built-in function to get uppercase names

SELECT UPPER(name) FROM student_info;

(Custom UDFs can be added in Java if required.)

[Link] an Index
CREATE INDEX idx_dept
ON TABLE student_info (dept)
AS 'COMPACT'
WITH DEFERRED REBUILD;
SHOW INDEXES ON student_info;

Rebuild the index:

ALTER INDEX idx_dept ON student_info REBUILD;

[Link] Operations
DROP INDEX idx_dept ON student_info;
DROP VIEW high_scorers;
DROP TABLE student_info;
DROP DATABASE college;
OUTPUT:
Sample Table Data:

1 John CS 85
2 Mary IT 92
3 Sam CS 70
4 Anna IT 88
5 Paul CS 95

View Output (high_scorers):

Mary IT 92
Anna IT 88
Paul CS 95

Describe Table Output:

col_name data_type
id int
name string
dept string
marks int
year int
RESULT:
Thus code has been executed and verified successfully.