Assignment Data mining
2 : سكشين Cyber security:محمد ممدوح السيد محمد قسم: االسم
Full report about types of distance
In machine learning, distance measures play a crucial role in various algorithms,
particularly in clustering, classification, and similarity search tasks. Here are some
common types of distance measures used:
1. Euclidean Distance: Calculates the straight-line distance between two points in
Euclidean space. It's widely used and applicable in many machine learning algorithms,
such as k-nearest neighbors (KNN) and k-means clustering.
2. Manhattan Distance (Taxicab or City Block Distance): Measures the distance
between two points by summing the absolute differences of their coordinates. It's
commonly used when movement is restricted to orthogonal paths, such as in grid-
based environments or feature spaces with categorical variables.
3. Minkowski Distance: Generalizes both Euclidean and Manhattan distances. It's
defined as the nth root of the sum of the absolute differences raised to the power of
n. When n=2, it becomes the Euclidean distance; when n=1, it becomes the Manhattan
distance.
4. Cosine Similarity: Measures the cosine of the angle between two vectors in a
multidimensional space. It's commonly used to measure similarity between
documents or text data and is robust to the scale of the vectors.
5. Hamming Distance: Calculates the number of positions at which corresponding
symbols differ in two equal-length sequences of symbols. It's mainly used for
categorical variables or binary feature vectors.
6. Jaccard Distance: Measures dissimilarity between sample sets, representing the size
of the intersection divided by the size of the union of the sample sets. It's often used
for text analysis or document clustering.
7. Mahalanobis Distance: Measures the distance between a point and a distribution,
taking into account the covariance structure of the data. It's useful when data are
correlated and have different variances.
These distance measures are crucial for determining similarity or dissimilarity between
data points, which is fundamental in various machine learning tasks, including
clustering, classification, recommendation systems, and anomaly detection. Choosing
the appropriate distance measure depends on the nature of the data and the specific
task at hand.
___________________________________
• Is it possible to choose centroids in a deterministic method?
Yes, centroids can be chosen deterministically. One method is k-means++,
which iteratively selects centroids based on their distance from existing
centroids, ensuring they are well-spaced.
• 2- Is it possible to get the mean for non-zero elements only?
Yes, you can calculate the mean for non-zero elements by summing them
and dividing by the count of non-zero elements.
• 3- Do you the scaling techniques efficiency ?
Scaling techniques like standardization or normalization can improve
efficiency by ensuring features are on a similar scale, aiding convergence
and preventing certain features from dominating the distance
calculations.
• 4- When does the algorithm stop?
The algorithm stops when either the centroids don't change significantly
between iterations or after a fixed number of iterations. Other stopping
criteria include reaching a threshold for the objective function or when
points no longer change clusters.
• 5- How to deal with big data? (PCA, LDA)
Dealing with big data in k-means can involve techniques like Mini-batch K-
means for processing subsets of data at a time or dimensionality
reduction techniques such as PCA (Principal Component Analysis) or LDA
(Linear Discriminant Analysis) to reduce the dimensionality before
clustering.
• 6- Which algorithm calculates the distances between the points of
dataset?
The algorithm that calculates distances between points in the dataset for k-
means is typically Euclidean distance. However, other metrics like
Manhattan distance or cosine similarity can also be used based on the
characteristics of the data.