Data Mining Functionalities Explained
Data Mining Functionalities Explained
Cosine similarity measures the cosine of the angle between two vectors. For x and y, cosine similarity is calculated as (1*2 + 1*2 + 1*2 + 1*2) / (sqrt(4)*sqrt(16)) = 1, indicating x and y are in the same direction, thus very similar. This measure helps in high-dimensional positivity when magnitude varies .
A supermarket chain could utilize data mining for market basket analysis to identify product associations for impactful promotions. Association analysis and classification might be essential functionalities for uncovering purchasing patterns. Simple statistical analysis or data queries may not uncover complex relationships such as non-obvious associations or predictions about future purchasing behaviors based on past data .
Characterization provides a summary of the general characteristics of a target class of data tuples, while discrimination compares the target class with one or more contrasting classes. For instance, in a retail database, characterization can summarize average spending behaviors of loyal customers, while discrimination could compare spending patterns between loyal and non-loyal customers to identify distinctive features .
Smoothing by bin means replaces data within a bin by the mean value of that bin. For the given age data, this reduces variance and highlights prominent trends by reducing noise, making it easier to identify overall patterns. Other techniques, such as smoothing by bin median or bin boundaries, may preserve central tendency or limit value range adjustments, respectively .
Jaccard distance focuses on the ratio of unmatching to total non-zero attributes and results in 1 due to no common presence. Euclidean distance considers actual magnitudes, resulting in sqrt(4)=2. Practical implications involve context; Euclidean is sensitive to scale, useful when magnitude matters, while Jaccard is suitable for presence-based binary attributes .
Min-max normalization typically scales data within a specified range [min, max], often [0, 1]. Z-score normalization generates a mean-centered distribution where values range based typically on standard deviations, theoretically unbounded. Normalization by decimal scaling scales data to fall within [-1, 1] based on the maximum absolute value .
Methods include data imputation using mean or median values, deletion of tuples with missing values, or prediction of missing values using algorithms. Imputation maintains dataset size but may introduce bias, while deletion simplifies data at the risk of losing valuable information. Predictive modeling is computationally intensive yet can be highly accurate if robust models are used .
Z-score normalization using mean absolute deviation is less sensitive to outliers than standard deviation, making it preferable for datasets with anomalous values or non-normally distributed data. This allows more representative scaling across the dataset, which might be skewed by extreme outliers if using standard deviation .
Bin means can mask outliers by compressing them toward bin averages. Alternative strategies like using interquartile range to flag outliers based on distance from typical range or employing machine learning models to detect anomalies better preserve outlier identification .
For nominal attributes, dissimilarity is calculated using a simple matching coefficient, counting mismatches. In contrast, dissimilarity for asymmetric binary attributes removes influence from the absence of both attributes, often computed using the Jaccard coefficient to reflect inherent asymmetry .