lOMoAR c PSD| 59306481

ABSTRACT
The objective of the project is the predict the house price by using regression
models. The dataset consist of 23 features and 21613 records. Among the 23
features, there are 18 integer type, 4 float type and 1 objective type. There are no
missing values in the dataset.

Regression models used are linear regression, KNN, SVR, decision tree, random
forest, Gradient boost. The different matrices like ‘R squared’, ‘Mean absolute
error’ and ‘Mean squared error’ are used to evaluate the different models.
Benchmark laid in compare with the final model is linear regression. But, mainly
considering the R squared value the final model could get almost 10% better
results compared to the benchmark model hence concluded to be the final model

Among all the models, Gradient boosting has performed well with trainset
accuracy of 81.9 and testset with 75.5.
 lOMoAR c PSD| 59306481

CHAPTER I
1. INTRODUCTION
1.1 Problem statement
A house value is simply more than location and square footage. Like the features
that make up a person, an educated party would want to know all aspects that give
a house its value. For example, you want to sell a house and you don’t know the
price which you can take — it can’t be too low or too high. To find house price
you usually try to find similar properties in your neighborhood and based on
gathered data you will try to assess your house price.

1.2 Objective:
To predict the price of house which is not too high or too low using regression
models. The project challenges to predict the final price of each home.

1.3 About the Dataset:

Dataset consist of 21613 records and 23 attributes with zero missing values. 23
attributes are:

1. Cid: a notation for a house

2. Day_hours: Date house was sold
3. Price: Price is prediction target
4. Room_bed: Number of Bedrooms/House
5. Room_bath: Number of bathrooms/bedrooms
6. Living_measure: square footage of the home
7. Lot_measure: Square footage of the lot
8. Ceil: Total floors (levels) in house
9. Coast: House which has a view to a waterfront
[Link]: Has been viewed
11. Condition: How good the condition is (Overall)
12. Quality: grade given to the housing unit, based on grading system
13.Ceil_measure: square footage of house apart from basement
14.Basement_measure: square footage of the basement
15. Yr_built: Built Year
16. Yr_renovated: Year when house was renovated
[Link]: zip
 lOMoAR c PSD| 59306481

[Link]: Latitude coordinate

[Link]: Longitude coordinate
20. Living_measure15: Living room area in 2015(implies-- some
renovations) This might or might not have affected the lot size area
21. Lot_measure15: lot size area in 2015(implies-- some renovations)
[Link]: Based on the quality of room
23. Total_area: Measure of both living and lot

1.4 Null values, data types and unique values of dataset

[Link] Variables Null_values D_type Unique
1
cid 0 int64 21436
2
Day hours 0 object 372
3
Price 0 int64 3625
4
Room_bed 0 int64 13
5
Room_bath 0 float64 30
6
Living_measure 0 int64 1038
7
Lot_measure 0 int64 9782
8
Ceil 0 float64 6
9
Coast 0 int64 2
10
Sight 0 int64 5
11
Condition 0 int64 5
12
Quality 0 int64 12
13
Ceil_measure 0 int64 946
14
Basement 0 int64 306
15
yr_built 0 int64 116
16
yr_renovated 0 int64 70
17
Zipcode 0 int64 70
 lOMoAR c PSD| 59306481

18
Latitude 0 float64 5034
19
longitude 0 float64 752
20
Living_measure15 0 int64 777
21
Lot_measure15 0 int64 8689
22
Furnished 0 int64 2
23
Total_area 0 int64 11163

 There are no missing values in the dataset

 There are no duplicated values
 There are 18 integer type, 4 float type and 1 object type

1.5. SYSTEM REQUIREMENTS

Hardware Requirements
Minimum:
• Processor: Intel i3 / AMD Ryzen 3 or equivalent
• RAM: 4 GB (basic model training & prediction)
• Storage: 250 GB HDD / SSD
• GPU: Not required, but useful for small-scale ML tasks (e.g., NVIDIA GeForce
MX series)
• OS: Windows 10 / Linux (Ubuntu 20.04+) / macOS
Recommended:
• Processor: Intel i5 / i7 or AMD Ryzen 5 / 7
• RAM: 8–16 GB (smooth ML model training, Pandas/Numpy handling large
datasets)
• Storage: 512 GB SSD (for datasets & faster processing)
• GPU: NVIDIA GPU with CUDA support (e.g., GTX 1650 / RTX 2060+) if deep
learning involved
• OS: Windows 10/11 (64-bit), Ubuntu 20.04+, or macOS Monterey+

Software Requirements
Operating System:
• Windows / Linux / macOS
Programming Language:
• Python 3.8+
IDE / Development Tools:
• Jupyter Notebook / JupyterLab
• PyCharm / VS Code / Spyder
 lOMoAR c PSD| 59306481

Libraries (Core ML + Prediction):

• Data Handling:
o NumPy
o Pandas
• Visualization:
o Matplotlib
o Seaborn
o Plotly (optional for interactive charts)
• Machine Learning:
o Scikit-learn (regression, classification, prediction models)
o Statsmodels (time-series forecasting, regression analysis)
• Deep Learning (if needed):
o TensorFlow / Keras
o PyTorch
• Prediction & Analytics:
o SciPy (statistical methods)
o Prophet (for time-series forecasting)
Database (Optional):
• SQLite / MySQL / PostgreSQL (for storing datasets & results)
Version Control:
• Git & GitHub / GitLab / Bitbucket
 lOMoAR c PSD| 59306481

CHAPTER II

2. Exploratory Data Analysis(EDA)

Exploratory Data Analysis is an approach in analyzing dataset to summarize their
main characteristics, often using statistical graphics and other data
visualization methods.

The primary motive of EDA is to:

i. Better understanding of the data

ii. Identifying various data patterns

iii. Handling the outliers

iv. Handling the missing value of dataset

v. Encoding the categorical variables

vi. Knowing duplication of record

vii. Finding the correlation between the attributes

2.1 Dropping attributes: There are attributes like cid, zip code, longitude
and latitude which are irrelevant in predicting the target variables. Hence after
dropping these attributes the dataset is
 lOMoAR c PSD| 59306481

2.2 Summary statistics:

Graphs show the form of the distribution of the data and are a very useful tool
in exploring a dataset. Besides graphs, statistics that summarize the distribution
of the data, are used to transform data into information. The five-number
summary, which forms the basis for a boxplot, is a good example of
summarizing data. The below table is summary statistics of the dataset
 lOMoAR c PSD| 59306481

Dayhours: 5 factor analysis is reflecting for this column

price: Our taget column value is in 75k - 7700k range. As Mean > Median, it
's Right-Skewed.

Room_bed: Number of bedrooms range from 0 - 33. As Mean slightly > Me

dian, it's slightly Right-Skewed.

Room_bath: Number of bathrooms range from 0 - 8. As Mean slightly < Me

dian, it's slightly Left-Skewed.
Living_measure: Square footage of house range from 290 - 13,540. As Mea
n > Median, it's Right-Skewed.
Lot_measure: Square footage of lot range from 520 - 16,51,359. As Mean al
most double of Median, it's highly Right-Skewed.
Ceil: Number of floors range from 1 - 3.5 As Mean ~ Median, it's almost No
rmal Distributed.
Coast: As this value represent whether house has waterfront view or not. It's
categorical column. From above analysis we got know, very few houses has
waterfront view.
Sight: Value ranges from 0 - 4. As Mean > Median, it's Right-Skewed
Condition: Represents rating of house which ranges from 1 - 5. As Mean >
Median, it's Right-Skewed

Quality: Representign grade given to house which range from 1 - 13. As Me

an > Median, it's Right-Skewed.

Ceil_measure: Square footage of house apart from basement ranges in 290 -

9,410. As Mean > Median, it's Right-Skewed.

Basement: Square footage house basement ranges in 0 - 4,820. As Mean hig

hlty > Median, it's Highly Right-Skewed.
yr_built: House built year ranges from 1900 - 2015. As Mean < Median, it's
Left-Skewed.

yr_renovated: House renovation year only 2015. So this column can be used
as Categorical Variable for knowing whether house is renovated or not.
 lOMoAR c PSD| 59306481

Lat: Latitude ranges from 47.1559 - 47.7776 As Mean < Median, it's Left-Sk
ewed.
Long: Longitude ranges from -122.5190 to -121.315 As Mean > Median, it's
Right-Skewed.
Living_measure15: Value ranges from 399 to 6,210. As Mean > Median, it'
s Right-Skewed.

Lot_measure15: Value ranges from 651 to 8,71,200. As Mean highly > Med
ian, it's Highly Right-Skewed.
Furnished: Representing whether house is furnished or not. It's a Categorica
l Variable
Total_area: Total area of house ranges from 1,423 to 16,52,659. As Mean is
almost double of Median, it's Highly Right-Skewed

 New attribute 'house_age is created by considering 'sold year' & 'yr_built'

2.3 Types of EDA

There are 3 types of EDA:

i) Univariate analysis
ii) Bivariate analysis

iii) Multivariate analysis

2.3.1 Univariate analysis:

This is simplest form of data analysis, where the data being analyzed consi
sts of just one variable. Since it’s a single variable, it doesn’t deal with cau
ses or relationships. The main purpose of univariate analysis is to describe
the data and find patterns that exist within it.
 lOMoAR c PSD| 59306481

Analyzing the Variables one by one:

[Link] : Dayhours
Changing the ‘Dayhours’ to ‘month year’

Number of values for ‘month year’ is

Groupby the ‘month year’ and ‘price’ and find the mean of price

So the time line of the sale data of the properties is from May-2014 to May-2015
and April month have the highest mean price
 lOMoAR c PSD| 59306481

[Link] : Price (Target Variable)

 Summary statistics of Price

 Distribution of price variable

price of the house is not symmetrical it is rightly skewed with outliers.

Price range for most of the house is under 2000000
 lOMoAR c PSD| 59306481

[Link]: Room_bed
 Number of bed rooms by using python function value_counts

 Graphical representation of room_bed

Majority of the house has 3 bedrooms followed by 4 bedrooms, 2 bedrooms,5

bedrooms,6 bedrooms,1 bedroom,7,8 and 0 bedrooms. The 33 bedroom what
we got in the above analysis is definitely an outlier it needs to be deleted.
 lOMoAR c PSD| 59306481

[Link] : Room_bath
 Summary statistics of Price

Number of bathrooms vary from 0 to 8

Graphical representation of bathrooms:

2.5 number of bathrooms found in majority of the houses followed by 1 bathroom.

[Link] Living measure (square footage of the home)

Summary statistics and amount of skewness for living measure

Living measure is the square footage of the house. Living measure varied from
290 [Link] to 13540 [Link]. There is no much deviation of mode from mean. Almost
 lOMoAR c PSD| 59306481

normal distribution with slightly right skewed. Skew factor is 1.47. From the
above graph it is observed that majority of the house's living measure is 2000 [Link]

Distribution of living measure from graph

Box plot for living measure

Many outliers are observed. It needs to be taken care in the future analysis.
 lOMoAR c PSD| 59306481

[Link] lot_measure (square footage of the lot)

Summary statistics and amount of skewness for lot measure

Box plot for lot measure

Lot measure varies from 520 to 16.5 Lakh square feet. Distribution is highly
skewed with outliers

[Link] ceil (Total floors in house)

Total number of floors in house by using value_counts

Total floors count by using graph

 lOMoAR c PSD| 59306481

Most of the house has 1 floor followed by 2 floors,1.5 and 3 floors

[Link] coast (House which has a view to a waterfront)

Value count of coast is

No coast and coast region count from graph is

 lOMoAR c PSD| 59306481

Coast is found binary type of data. House which have waterfront view represents
1 and house without waterfront view represents 0. Only 163 houses has waterfront
view.

[Link] Sight (represents how many times sight has been viewed)
Value counts for sight is

Range of the sight varies from 0 to 4. Most of the house have not been viewed.
And the respective values of how many times its been viewed is as shown

[Link] : Condition (Overall condition of the house)

Value count for condition of house is

Overall condition of the house is represented in the scale of 1 to 5. Majority of

the house falls in the category of 3

[Link] : Quality (grade given to the housing unit, based on

grading system)
Value counts for quality which is based on grading system
 lOMoAR c PSD| 59306481

Quality - most properties have quality rating between 6 to 10

There are only 13 properties which have the highest quality rating

[Link] : Ceil_measure (square footage of house apart from

basement)
 Graphical representation of ceil measure

 Value_count and skewness for Ceil_measure is

 lOMoAR c PSD| 59306481

Relation between ceil and ceil_measure

Found there are no direct or linear relation between ceil and ceil_measure

[Link] : Basement_measure (square footage of the basement)

 Skewness and distribution for basement_measure is

If skewness is between -0.5 and 0.5, the distribution is approximately

symmetric
As the skewness is greater than 1, the distribution is extremely skewed
 lOMoAR c PSD| 59306481

2 distribution can be seen. That means there are some properties without
basement and some are with basement.

 Value counts for basement measure:

More than 50% of the property doesn't have basement

Boxplot for basement_measure

 lOMoAR c PSD| 59306481

Boxplot for property which has basement only. We can see that there are many
outliers that needs to addressed.

Distribution of houses with basement only

Distribution having basement is right-skewed

 lOMoAR c PSD| 59306481

[Link] : Yr_built (Built Year)

 The below graph is the distribution of yr_built

Value counts for yr_built is

variation of yr_built can be observed from 1900 to 2014, With more properties
built in the year 2014

[Link] : Yr_renovated (Year when house was renovated)

Summary statistic of yr_renovated
 lOMoAR c PSD| 59306481

914 houses were renovated

Distribution of renovated houses:

[Link] : Furnished (Based on the quality of room)

Value_counts of house in which 0 as furnished and 1 as not furnished
 lOMoAR c PSD| 59306481

2.3.2 Bivariate analysis

Pairplot:
To plot multiple pairwise bivariate distributions in a dataset, you can use the
pairplot() function. This shows the relationship for (n, 2) combination of variable
in a Dataframe as a matrix of plots and the diagonal plots are the univariate plots
 lOMoAR c PSD| 59306481

From above pair plot, we observed/deduced below

1. price: price distribution is Right-Skewed as we deduced earlier from our 5-factor

analysis

2. room_bed: our target variable (price) and room_bed plot is not linear. Its
distribution have lot of gaussians

3. room_bath: It's plot with price has somewhat linear relationship. Distribution
has number of gaussians.

4. living_measure: Plot against price has strong linear relationship. It also have
linear relationship with room_bath variable. So might remove one of these 2.
Distribution is Right-Skewed.

5. lot_measure: No clear relationship with price.

 lOMoAR c PSD| 59306481

6. ceil: No clear relationship with price. We can see, it's have 6 unique values only.
Therefore, we can convert this column into categorical column for values.

7. coast: No clear relationship with price. Clearly it's categorical variable with 2
unique values.

8. sight: No clear relationship with price. This has 5 unique values. Can be
converted to Categorical variable.

9. condition: No clear relationship with price. This has 5 unique values. Can be
converted to Categorical variable

10. quality: Somewhat linear relationship with price. Has discrete values from 1 -
13. Can be converted to Categorical variable.

11. ceil_measure: Strong linear relationship with price. Also with room_bath and
living_measure features. Distribution is Right-Skewed.

12. basement: No clear relationship with price.

13.yr_built: No clear relationship with price.

14. yr_renovated: No clear relationship with price. Have 2 unique values. Can be
converted to Categorical Variable which tells whether house is renovated or not.

15. zipcode, lat, long: No clear relationship with price or any other feature.

16. living_measure15: Somewhat linear relationship with target feature. It's same
as living_measure. Therefore we can drop this variable.

17. lot_measure15: No clear relationship with price or any other feature.

18. furnished: No clear relationship with price or any other feature. 2 unique values
so can be converted to Categorical Variable

19. total_area: No clear relationship with price. But it has Very Strong linear
relationship with lot_measure. So one of it can be dropped.

 In brief, below features should be converted to Categorical Variable

ceil, coast, sight, condition, quality, yr_renovated, furnished

 Below columns can be dropped after checking pearson factor

zipcode, lat, long, living_measure15, lot_measure15, total_area, sold_year

 lOMoAR c PSD| 59306481

Correlation between independent variables and target variable

We have linear relationships in below features as we got to know from above

matrix

1. price: room_bath, living_measure, quality, living_measure15, furnished

2.living_measure: price, room_bath. So we can consider dropping 'room_bath'

variable

[Link]: price, room_bath, living_measure.

4.ceil_measure: price, room_bath, living_measure, quality.

5. living_measure15: price, living_measure, quality. So we can consider

dropping living_measure15 as well. As it's giving same info as living_measure.

6. lot_measure15: lot_measure. Therefore, we can consider dropping

lot_measure15, as it's giving same info.

7. furnished: quality
 lOMoAR c PSD| 59306481

8. total_area: lot_measure, lot_measure15. Therefore, we can consider dropping

total_area feature as well. As it's giving same info as lot_measure.
Heatmap
A heatmap is a graphical representation where individual values of a matrix are
represented as colors. A heatmap is very useful in visualizing the concentration of
values between two dimensions of a matrix. This helps in finding patterns and
gives a perspective of depth.

Room_bed v/s price

 lOMoAR c PSD| 59306481

With increasing in room bed 'Price' is increasing. After 8 room_bed the graph is
following decreasing trend

Room_bath v/s price

lOMoAR c PSD| 59306481
 lOMoAR c PSD| 59306481

Price is following upward trend with respect to room_bath

Living_measure v/s Price (Square foot of home)

There is a clear linear relationship can be observed with respect to price and
living_measure. Also few outliers can be visible

Lot_measure v/s Price(Square foot of lot)

 lOMoAR c PSD| 59306481

lot_measure-there seems to be no relation between lot_measure and price

lot_measure - data value range is very large so breaking it get better view.

No clear trend is observed between price and lot_measure. Also the correlation
between price and lot_measure is 0.082456. We could drop this feature

Ceil v/s Price (Number of floors)

Mean, median and size for price with respect to ceil

ceil - median price increases initially and then falls. There is some slight upward
trend in price with the ceil
 lOMoAR c PSD| 59306481

Coast v/s Price

Mean and median for price with respect to coast

coast - mean and median of waterfront view is high however such houses are very
small in compare to non-waterfront view. Also, living_measure mean and median
is greater for waterfront house. The house properties with water_front tend to
have higher price compared to that of non-water_front properties.

Sight v/s Price

Mean and median for price with respect to sight
 lOMoAR c PSD| 59306481

sight - have outliers. The house sighted more have high price (mean and median)
and have large living area as well. Properties with higher price have more no of
sights compared to that of houses with lower price.

condition v/s Price

Mean and median for price with respect to condition
 lOMoAR c PSD| 59306481

condition - as the condition rating increases its price and living measure mean and
median also increases. The price of the house increases with condition rating of the
house

Quality v/s Price

Mean and median for price with respect to quality
 lOMoAR c PSD| 59306481

quality - with grade increase price and living_measure increase (mean and
median). There is clear increase in price of the house with higher rating on quality

Ceil_measure v/s Price

ceil_measure - price increases with increase in ceil measure. There is upward trend
in price with ceil_measure.

Basement v/s price

 lOMoAR c PSD| 59306481

We will create the categorical variable for basement 'has_basement' for houses
with basement and no basement. This categorical variable will be used for further
analysis
 lOMoAR c PSD| 59306481

The houses with basement has better price compared to that of houses without
basement.

yr_renovated v/s Price

mean and median for price with respect to year renovated
 lOMoAR c PSD| 59306481

So most houses are renovated after 1980's. We will create new categorical
variable 'has_renovated' to categorize the property as renovated and non-
renovated. For further analysis we will use this categorical variable.

House age vs price

Analyzing house age with respect to price
 lOMoAR c PSD| 59306481

Trend is not clear with respect to age of the house and Price. Correlation between
them is also very low.
furnished v/s Price
Mean and median of price with respect to furnished

furnished - Furnished has higher price value and has greater living_measure.
Furnished houses have higher price than that of the Non-furnished houses
 lOMoAR c PSD| 59306481

CHAPTER III
3. Data Pre-processing
Data preprocessing is the process of transforming raw data into an
understandable format. It is also an important step in data mining as we cannot
work with raw data. The quality of the data should be checked before applying
machine learning or data mining algorithms.
3.1 Outliers handling
We have seen outliers in many of the attributes like columns room_bath,
living_measure, lot_measure, ceil_measure and Basement

3.2 Treating outliers for column - ceil_measure

Lower bound and upper bound values ranges from -34 to 3740

Let us have 3 dataframes

1. hdf: without outliers (Its outliers are found and dropped)

2. hdf_log_trans: The columns where outliers are found that particular columnis
log transformed

3. hdf_with_outlier: The dataframe with outliers.

All the three dataframe have been considered for model building seperately
 lOMoAR c PSD| 59306481

Dropping the outliers records from dataset

All the outliers values in both upper and lower limit have been removed

Let us analyse by log transform

Only 'ceil_measure_log' is considered and the original column have been dropped

After log transformation distribution found almost normal

 lOMoAR c PSD| 59306481

Treating outliers for column - living_measure

Now let us consider the log transformation:

Plot a distribution plot for living measure and living measure log
 lOMoAR c PSD| 59306481

Skewness after dropping outlier is

Treating outliers with column-lot_measure

Lower bound_lom and upper bound_lom outlier treatment

Dataset with lot_measure less than lowerbound_lom and greater than

upperbound_lom

We got 2128 records which are outliers. Drop these records from dataset

Log transformation

Plot for lot_measure and lot_measure_log

 lOMoAR c PSD| 59306481

Dropping the unwanted columns

From the data analysis it is found that ‘cid’, ‘month_year’, ‘year renovated’, ‘zip
code’, ‘latitude’ and ‘longitude’ are irrelevant in predicting the target
variable(Price), so need to drop them

Final dataset
 lOMoAR c PSD| 59306481

For the convenience lets rename dataframes as

df0 ------> with outliers

df ------ > without outliers (Outliers have been dropped)

df1 ------ > Log transformed (Some of the features only)

One hot encoding:

It is a technique to treat categorical variables. It simply creates additional features
based on number unique values in categorical features.
Every unique value in the category will be added as a feature
One hot encoding is the process of creating dummy variables
Creating dummies for categorical variables: 'room_bed', 'room_bath', 'ceil',
'coast', 'sight', 'condition', 'quality', 'furnished', 'has_basement'.

Final dataframes for modelling

Dataframe shape without outliers is

Dataframe shape with outliers is

Dataframe shape with log transformation

 lOMoAR c PSD| 59306481

CHAPTER IV

4. Data modelling
Machine learning model is the product of training a machine learning algorithm
with training data.
While algorithms are simply general approaches to solve an objective, ML
models can evaluate future unknown data and make predictions.
One can create many models from same algorithm, as long as different training
data are available.
Types of ML algorithms
1. Supervised learning:
2. Unsupervised learning:

Supervised learning: Supervised learning can be further grouped into regression

and classification:
• Regression: Regression problem is when the output variable is real
value, such as number of students in classroom, weight of person
Ex: Linear regression, KNN regressor, support vector regressor,
Decision tree, random forest
• Classification: A classification problem is when the output variable is
a category, such as ‘red’ or ‘blue’ color, has ‘disease’ or ‘no disease’
Ex: Logistic regression, Decision tree, random forest

The target variable ‘Price’ is a real value and is an example of supervised

regression problem

Important libraries to be used in the project

 lOMoAR c PSD| 59306481

Four different dataframes has been made for modelling.

4. Original dataframe with outliers

5. Dataframe without outliers
6. Dataframe with log transformed columns
7. Scaled dataframe

Function is defined to calculate model performance of given model and

dataset
 lOMoAR c PSD| 59306481

Three performance metrics is considered to evaluate the model.

8. R squared
9. Mean squared error
[Link] absolute error

4.1 Linear regression (Benchmark model)

Performance of the model with outlier, without outlier, and log transformation

Model performance for with outliers and log transformation for training data is
more and it is 0.706 and 0.708

Model performance for with outliers and log transformation for test data is more
and it is 0.664 and 0.667
 lOMoAR c PSD| 59306481

Model for data with scaling:

Scaling is a method used to normalize the range of independent variables or
features of data

• Linear regression model gave consistent result without overfitting or

underfitting for overall data and data with log transformed columns
• Linear regression model overfits and performed very poor for data with
outliers and for scaled data

4.2 KNN Regressor

Model performance for with outliers, without outliers and for log transformation
 lOMoAR c PSD| 59306481

• KNN regressor overfits for all the dataframes

• Best model with KNN regressor is achieved for scaled data

4.3 Support vector regressor

Model performance for with outliers, without outliers and for log transformation
 lOMoAR c PSD| 59306481

Support vector machine regressor is fitting worse than horizontal line for all the
dataframes. Hence SVM model is not suitable for the present data.

4.4 Decision tree

Model performance for with outliers, without outliers and for log transformation

• Decision tree regressor worked consistently with all dataframes without any
overfitting or underfitting.
• Decision tree gave better result for overall data and bad result for scaled data.

4.5 Random forest

Model performance for with outliers, without outliers and for log transformation
 lOMoAR c PSD| 59306481

• Random forest regressor gave better and consistant result without overfitting
or underfitting.
• Model performance is better for overall data without dropping out outliers.

• Out of all the algorithms, Linear regression for overall original data gave a
better performance hence that can be considered to be a better and final model.

Performance evaluation metrics of all algorithms

 lOMoAR c PSD| 59306481

Improving Performance of all these models will be done by fine tuning the models
by using cross validation, random searchCV and hyperparameters.

4.6 Gradient boosting

Model performance with outliers, without outliers, Log transformation and with
scaling

Gradient Boost Regressor algorithm is giving a consistent and better result compared to all other
models. Hence, here after Gradient Boost Regressor with log transformation data will be
considered for further analysis

Performance evaluation metrics of all algorithms

 lOMoAR c PSD| 59306481

4.7 Feature importance

It refers to the technique that assign a score to input features based on how useful
they are at predicting a target variable.
There are many types and sources of feature importance scores, although popular
examples include statistical correlation scores, coefficients calculated as part of
linear models, decision trees, and permutation importance scores.
 lOMoAR c PSD| 59306481

First 20 features have an importance of 98.037 and first 30 have importance of

99.445

1. The ensemble models (Random Forrest and Gradient Boost Regressor) have
performed well compared to that of linear, KNN, SVR models.

2. The best performance is given by Gradient boosting model with training score-
82%, Testing score 76%

3. The top key features that drive the price of the property
are:'living_measure_log','furnished_1','house_age','living_measure15','coast_1','
yr_built' etc...
 lOMoAR c PSD| 59306481

4.8 Pickling the model

Pickling is a useful python tool that allows you to save your models, to minimize
lengthy re-training and allow you to share, commit and reload pre-trained
machine learning models

After removing few variables which are not relevant in predicting the target, the
final independent variables are

4.9 Build Gradient boosting regressor for final dataset

since there are only 20 attributes. Let us build our GBR model for this data:

Since it is learnt that the best model is Gradient Boost for our regression problem. Also, the
hyperparameter tuning is done by [Link] best model with best hyperparameter and the
 lOMoAR c PSD| 59306481

best data is found from our analysis. And the same is considered for model building without one
hot encoding.

4.11 Feature importance

 lOMoAR c PSD| 59306481

4.12 Pickling the model

Pickling The final Gradient Boost Regressor model without considering one hot
encoding:

Best model is pickled and crosschecked.

4.13 Checking the price of house by inputing different values for

independent attributes
House price prediction for new data point
 lOMoAR c PSD| 59306481

Model evaluation
Many models were built for different data sets before concluding the best model
for the given problem as explained in the previous section in detail.
At the end of numerous trials carried out on the different data sets and the models,
the gradient boosting regressor is selected as the final model for the current
project.

The objective of the model in the present study was to predict the house price
given many features or attributes about the house as accurately as possible so that
the difference between actual and predicted price is minimized. Hence the
problem was regression problem predicting a real valued number as an output.
To achieve the above objective various models were tried and at last the gradient
boosting algorithm with 200 numbers of estimators and 0.1 as a learning rate gave
the best result out of the lot and hence concluded to be the final model for the
present case.

The output of the model confirmed that the features like ‘living measure’,
‘furnished’ and ‘house age’ were most prominent features in predicting the house
price along with many other parameters.

To decide the best model among the lot, the models were compared with the
benchmark model ‘Linear regression’ performance using ‘R squared’, ‘Mean
absolute error’ and ‘Mean squared error’ as a evaluation matrices.

Comparison to benchmark
In the present study, instead of keeping the particular value as a benchmark, the
basic linear regression model performance was considered to be a benchmark and
 lOMoAR c PSD| 59306481

further different models were built and trained in order improve the results
compared the results of the benchmark model. The benchmark ‘Linear
regression’ model could get us around 65% accurate result in predicting the house
prices.
The different matrices like ‘R squared’, ‘Mean absolute error’ and ‘Mean squared
error’ are used to evaluate the different models. But, mainly considering the R
squared value the final model could get almost 10% better results compared to
the benchmark model hence concluded to be the final model.
The different models performed differently on all the datasets but finally the
ensembles model with proper hyper parameter tuning could give better results in
predicting the house price hence could improve the results compared to
benchmark model results. The better performance could be due to
accommodating the non-linear behaviour in the model which was not present in
the benchmark model and also the generalisation of results of many models due
to the use of ensemble technique in the form of gradient boosting has helped
reduce the bias and variance without losing maximum feature importance and
information from the data. Hence the final model could perform better compared
to the benchmark set.
Visualizations
In addition to quantifying the model and the solution, the below are the relevant
visualizations that support the ideas/insights that gleaned from the data.

Feature importances:
 lOMoAR c PSD| 59306481

Heatmap

Bivariate analysis
 lOMoAR c PSD| 59306481

Implications
We have arrived at our final model which is able to predict the price of a house with 95% of
confidence interval. Our final model train accuracy is 82% and test accuracy is 76%.

Table showing price of the house (actual v/s predicted)

From the above observation it can be seen that a random cross checking is made showing the
comparison of our model's predicting price v/s actual price of the house. Our model’s prediction
can be found in line with the actual price of the house tested for different records.

Limitations
The main limitations of our solution are,

• Treating outliers: there can be found quite numerous outliers in many attributes.
Removing outliers yielded drop in the model accuracy in almost all the algorithms we
tried.

• Handling categorical variable: for the same, we tried one hot encoding which results
in the increase of number of attributes. That would be very difficult to input the different
attributes when it is taken to production environment.

• Visualization: Since, the number of independent attributes is high the visualization and
drawing the relation between each attribute were not easy.
 lOMoAR c PSD| 59306481

We have found outliers individually for each attribute and taken care for the same.
For the final model we have tested by removing dummy variables and the model
behaved smoothly without change in its performance and accuracy. For
visualization we tried univariate, bivariate and multivariate plots and tried to
extract maximum relation among independent variable and between independent
and target variable.
 lOMoAR c PSD| 59306481

CHAPTER V
5. CODING

import pandas as pd
import numpy as np
import [Link] as plt
from [Link] import scatter_matrix
import seaborn as sns
from scipy import stats
from sklearn import preprocessing
from [Link] import OneHotEncoder

cal_housing = pd.read_csv('[Link]')

# 1 Explore Data

print(cal_housing.info())
print(cal_housing.head())
print(cal_housing.describe())
print(cal_housing["ocean_proximity"].value_counts())

# 2 Scaling

# for the normalizer

median_value = cal_housing['total_bedrooms'].median()
cal_housing['total_bedrooms'].fillna(median_value, inplace=True)
number_columns = ['longitude', 'latitude', 'housing_median_age',
'total_rooms', 'total_bedrooms', 'population', 'households',
'median_income', 'median_house_value']
cal_housing_numbers = cal_housing[number_columns]

min_max_test = [Link]()
X_train_minmax_test = min_max_test.fit_transform(cal_housing_numbers)
cal_housing_minmax_scaled_df = [Link](X_train_minmax_test,
columns=number_columns)
print(cal_housing_minmax_scaled_df.head())

standar_test = [Link]()
X_train_standar_test = standar_test.fit_transform(cal_housing_numbers)
cal_housing_standard_scaled_df = [Link](X_train_standar_test,
columns=number_columns)
print(cal_housing_standard_scaled_df.head())

maxabs_test = [Link]()
X_train_maxabs_test = maxabs_test.fit_transform(cal_housing_numbers)
cal_housing_maxabs_scaled_df = [Link](X_train_maxabs_test,
columns=number_columns)
print(cal_housing_maxabs_scaled_df.head())

robust_test = [Link]()
X_train_robust_test = robust_test.fit_transform(cal_housing_numbers)
cal_housing_robust_scaled_df = [Link](X_train_robust_test,
columns=number_columns)
print(cal_housing_robust_scaled_df.head())
 lOMoAR c PSD| 59306481

normal_test = [Link]()
X_train_normal_test = normal_test.fit_transform(cal_housing_numbers)
cal_housing_minmax_scaled_df = [Link](X_train_normal_test,
columns=number_columns)
print(cal_housing_minmax_scaled_df.head())

# 3 One hot Vector

unique_ocean_proximity = cal_housing['ocean_proximity'].unique()
print(unique_ocean_proximity)

ohv_categories = cal_housing[["ocean_proximity"]]
encoder = OneHotEncoder()
housing_cat_ohv = encoder.fit_transform(ohv_categories)
array = housing_cat_ohv.toarray()
encoderDF = [Link](array, columns=encoder.categories_)
dataset = [Link]([cal_housing, encoderDF], axis=1)
dataset = [Link](['ocean_proximity'], axis=1)
print(dataset)

# 4 median value

print(cal_housing.info())
print("Κενές τιμές πριν: ")
print(cal_housing['total_bedrooms'].isna().sum())
median_value = cal_housing['total_bedrooms'].median()
cal_housing['total_bedrooms'].fillna(median_value, inplace=True)
print("Κενές τιμές μετά: ")
print(cal_housing['total_bedrooms'].isna().sum())

# visualize

cal_housing.hist(bins=50, figsize=(15,8.1))
[Link]()

features = ['longitude', 'latitude', 'housing_median_age', 'total_rooms',

'total_bedrooms', 'population', 'households', 'median_income',
'median_house_value']
for feature in features:
[Link](cal_housing[feature], bins=50, density=True, alpha=0.6,
color='b')
[Link](f'Histogram of {feature}')
[Link](feature)
[Link]('Probability density')
[Link]()

for feature in features:

 lOMoAR c PSD| 59306481

[Link](cal_housing[feature], kde=True, color='b')

[Link](f'Histogram of {feature}')
[Link](feature)
[Link]('Probability density')
[Link]()

# visual 2

cal_housing.plot(kind="scatter", x="longitude", y="latitude", alpha=0.4,

s=cal_housing["population"]/100, label="population", figsize=(10,7),
c="median_house_value", cmap=plt.get_cmap("jet"), colorbar=True,
sharex=False)
[Link]('California housing prices')
[Link]()
[Link]()

[Link](cal_housing["median_income"],
cal_housing["median_house_value"], alpha=0.1)
[Link]("Median Income")
[Link]("Median House Value")
[Link]("Median Income per Median House Value")
[Link]([0, 16, 0, 550000])
[Link]()

[Link](cal_housing["total_rooms"], cal_housing["median_house_value"],
alpha=0.1)
[Link]("Total Rooms")
[Link]("Median House Value")
[Link]("Total Rooms per Median House Value")
[Link]()

# Perceptron

from perceptron import Perceptron

if __name__ == "__main__":

cal_housing = pd.read_csv('[Link]')

def accuracy(y_true, y_pred):

accuracy = [Link](y_true == y_pred) / len(y_true)
return accuracy

def mean_squared_error(y_true, y_pred):

mse = [Link]((y_true - y_pred) ** 2)
return mse

def mean_absolute_error(y_true, y_pred):

mae = [Link]([Link](y_true - y_pred))
return mae
 lOMoAR c PSD| 59306481

X = cal_housing.iloc[:, :-2].values
y = cal_housing.iloc[:, -2].values

k = 5
fold_size = len(X) // k

accuracies = []
mses = []
maes = []

# k-fold cross validation

for i in range(k):
test_start = i * fold_size
test_end = (i + 1) * fold_size
train_indices = list(range(test_start)) + list(range(test_end,
len(X)))

X_train, X_test = X[train_indices], X[test_start:test_end]

y_train, y_test = y[train_indices], y[test_start:test_end]

p = Perceptron(learning_rate=0.01, n_iters=1000)
[Link](X_train, y_train)
predictions = [Link](X_test)

acc = accuracy(y_test, predictions)

mse = mean_squared_error(y_test, predictions)
mae = mean_absolute_error(y_test, predictions)

[Link](acc)
[Link](mse)
[Link](mae)

mean_accuracy = [Link](accuracies)
mean_mse = [Link](mses)
mean_mae = [Link](maes)

print("Mean classification accuracy:", mean_accuracy)

print("Mean Squared Error:", mean_mse)
print("Mean Absolute Error:", mean_mae)

# least squares

def kfold_indices(N, K):

indices = [Link](N)
M = N // K
if N % K != 0:
raise ValueError("The number of elements within vector Indices must
be fully divided by K")
else:
train_indices = []
test_indices = []
for k in range(K):
 lOMoAR c PSD| 59306481

start = k * M
end = (k + 1) * M if k < K - 1 else N
test_indices.append(indices[start:end])
train_indices.append(np.setdiff1d(indices, test_indices[-1]))
return train_indices, test_indices

def linear_regression_fit(X, Y, K=10):

mse_scores = []
mae_scores = []

train_indices, test_indices = kfold_indices(len(X), K)

for train_index, test_index in zip(train_indices, test_indices):

X_train, X_test = [Link][train_index], [Link][test_index]
Y_train, Y_test = [Link][train_index], [Link][test_index]

X_train_mean = [Link](X_train)
Y_train_mean = [Link](Y_train)

num = [Link]((X_train - X_train_mean) * (Y_train - Y_train_mean))

den = [Link]((X_train - X_train_mean) ** 2)
m = num / den
c = Y_train_mean - (m * X_train_mean)

Y_pred = m * X_test + c

mse = [Link]((Y_test - Y_pred) ** 2)

mae = [Link]([Link](Y_test - Y_pred))

mse_scores.append(mse)
mae_scores.append(mae)

[Link](X_test, Y_pred, color='red')

return mse_scores, mae_scores

if __name__ == "__main__":

X = cal_housing.iloc[:, 0]
Y = cal_housing.iloc[:, 1]

[Link](X, Y)

mse_scores, mae_scores = linear_regression_fit(X, Y)

print("Mean Squared Error (MSE) Scores:", mse_scores)

print("Mean Absolute Error (MAE) Scores:", mae_scores)
print("Average MSE:", [Link](mse_scores))
print("Average MAE:", [Link](mae_scores))

[Link]("X")
[Link]("Y")
[Link]("Linear Regression Fit")
[Link]()
 lOMoAR c PSD| 59306481

# MultiLayer NN

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from [Link] import StandardScaler
from [Link] import Sequential
from [Link] import Dense
from [Link] import Adam
from [Link] import EarlyStopping
import [Link] as plt
from [Link] import r2_score

# Τα ξανακάνω απλά για να φανούν και εδώ

dataset = pd.read_csv('[Link]')
median_value = dataset['total_bedrooms'].median()
dataset['total_bedrooms'].fillna(median_value, inplace=True)
dataset = pd.get_dummies(dataset, columns=['ocean_proximity'])

X = [Link][:,0:9]
Y = [Link][:,9]

# Split the data into training and validation sets

X_train, X_val, Y_train, Y_val = train_test_split(X, Y, test_size=0.3,
random_state=42)

# Standardize features by removing the mean and scaling to unit variance

scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_val = [Link](X_val)

# Create model
model = Sequential([
Dense(128, activation="relu", input_dim=X_train.shape[1]),
Dense(32, activation="relu"),
Dense(8, activation="relu"),
Dense(1, activation="linear")
])

# Compile model
[Link](loss='mean_squared_error', optimizer=Adam(learning_rate=1e-3,
decay=1e-3 / 200))

# Patient early stopping

es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=200)

# Fit the model

history = [Link](X_train, Y_train, validation_data=(X_val, Y_val),
epochs=10000000, batch_size=100, verbose=2, callbacks=[es])

# Calculate predictions
PredTestSet = [Link](X_train)
PredValSet = [Link](X_val)
 lOMoAR c PSD| 59306481

# Plot loss history

[Link]([Link]['loss'], label='train')
[Link]([Link]['val_loss'], label='test')
[Link]('Trainig History')
[Link]('Epoch')
[Link]('Validation loss')
[Link]()
[Link]()

# Compute R-Square value for validation set

ValR2Value = r2_score(Y_val, PredValSet)
print("Validation Set R-Square=", ValR2Value)

import numpy as np

class Perceptron:
def __init__(self, learning_rate=0.01, n_iters=1000):
[Link] = learning_rate
self.n_iters = n_iters
self.activation_func = self._unit_step_func
[Link] = None
[Link] = None

def fit(self, X, y):

n_samples, n_features = [Link]

[Link] = [Link](n_features)
[Link] = 0

y_ = [Link]([1 if i > 0 else 0 for i in y])

for _ in range(self.n_iters):

for idx, x_i in enumerate(X):

linear_output = [Link](x_i, [Link]) + [Link]

y_predicted = self.activation_func(linear_output)

update = [Link] * (y_[idx] - y_predicted)

[Link] += update * x_i

[Link] += update

def predict(self, X):

linear_output = [Link](X, [Link]) + [Link]
y_predicted = self.activation_func(linear_output)
return y_predicted

def _unit_step_func(self, x):

return [Link](x >= 0, 1, 0)
 lOMoAR c PSD| 59306481

SCREENSHOT
lOMoAR c PSD| 59306481
lOMoAR c PSD| 59306481
lOMoAR c PSD| 59306481
lOMoAR c PSD| 59306481
lOMoAR c PSD| 59306481
lOMoAR c PSD| 59306481
lOMoAR c PSD| 59306481
lOMoAR c PSD| 59306481
 lOMoAR c PSD| 59306481

CHAPTER VI
6. Conclusions
We have built different models on the datasets with all possible
hyperparameter tuning. Ensembles models particularly Gradient Boost
Regressor is performing better with good accuracy and performance score.
The final best model is pickled for further use and exposed as rest API using
‘flask’ framework which is able to predict the price of the property in the
local machine.

The top key features to consider for pricing a property are:

’quality’,’living_measure,’house age’, ‘Coast’, ‘furnished', 'yr_built'. So, one
needs to thoroughly introspect its property on parameters suggested and list
its price accordingly, similarly if one wants buy house - needs to check the
features suggested above in house and calculate the predicted price. The same
can then be compared to listed price.

For further improvisation, the datasets can be made by treating outliers in

different ways. Making polynomial features and improvising the model
performance can also be explored further. The scope for future work is model
is pickled and exposed as rest API it can be deployed in the external server
using AWS or Heroku cloud platform.
 lOMoAR c PSD| 59306481

REFERENCE

Εισαγωγή στην Αναγνώριση Προτύπων με MATLAB, S. Theodoridis, A. Pikrakis, K. Koutroumbas,

D. Cavouras, 2011, Εκδόσεις Broken Hill Publishers Ltd.
[2] [Link]
[3] [Link]
[4] [Link]
[5] [Link]