Data Science Exam Question Bank
Uploaded by
RohitData Science Exam Question Bank
Uploaded by
RohitCommon questions
Powered by AIEDA is critical as it allows data scientists to understand the underlying patterns, spot anomalies, test hypotheses, and check assumptions through graphical representations and summary statistics. It helps in identifying the structure and patterns relevant to the research problem. By exploring the data in the early stages, scientists can make better decisions on data pre-processing and the choice of analysis methods or modeling techniques. EDA complements subsequent analysis processes by ensuring data quality and enhancing hypothesis testing and modeling accuracy through initial insights gained .
Descriptive statistics provide a summary of the data by identifying central tendencies, dispersion, and distribution shapes. EDA extends this by visually exploring data patterns and relationships. While descriptive statistics offer a numerical basis for understanding data, EDA provides the visual context necessary to identify trends, detect outliers, and uncover hidden patterns. Together, they help in forming hypotheses and strategies for further detailed analysis .
Data visualization aids in EDA by making complex datasets more accessible and understandable through visual tools, thereby facilitating quick identification of patterns, outliers, and trends. Common visualization types include histograms, scatter plots, box plots, and bar charts. These tools allow analysts to visually assess the shape, spread, and relationship between variables, crucial for forming hypotheses and guiding further analysis .
Data Cleaning provides a foundation for accurate data-driven decisions by ensuring the data is free from errors, inconsistencies, or missing information. Common methods include handling missing data through imputation or deletion, correcting inaccuracies, removing duplicates, and ensuring the data is formatted consistently. Without proper data cleaning, analyses might be based on flawed data, leading to potentially incorrect conclusions .
Quantitative data is numerical and can be measured and counted, while qualitative data is descriptive and represents characteristics that cannot easily be measured. This distinction affects the choice of analysis methods: quantitative data is typically subject to statistical models, whereas qualitative data often involves categorical analysis and content interpretation. The distinction is crucial for selecting appropriate methods to analyze and interpret data effectively .
A Type I error occurs when a true null hypothesis is incorrectly rejected, commonly resulting in a false positive conclusion. A Type II error happens when a false null hypothesis is not rejected, leading to a false negative. Both errors impact decision-making; a Type I error could lead to unnecessary actions or changes, while a Type II error may result in missed opportunities or failure to act on valid findings .
PMF applies to discrete random variables and gives the probability that a discrete random variable is exactly equal to a given value. PDF applies to continuous random variables and describes the probability of the random variable falling within a particular range of values. Both functions describe the distribution of probabilities and must satisfy the properties of normalization (the total probability is 1).
Structured data is highly organized and easily searchable in databases; it is used in applications requiring complex queries and analytics, such as financial databases. Unstructured data, such as emails or social media posts, lacks strict format and requires sophisticated techniques like natural language processing for analysis. Semi-structured data, exemplified by JSON or XML files, combines elements of both. The challenges include managing the volume and variety (for unstructured data), integration into structured systems (for semi-structured data), and preserving data quality and consistency (for structured data).
Data preprocessing is vital as it prepares the raw data for analysis by cleaning, transforming, and organizing it to overcome inconsistencies, missing values, and invalid entries that could affect analysis. Common techniques include normalization, categorization, handling missing data, and encoding variables. This step ensures the accuracy and reliability of the data, ultimately enhancing model performance and decision outcomes .
Hypothesis testing is important as it provides a structured framework for making data-driven decisions about population parameters based on sample data. For example, in healthcare, a hypothesis test could evaluate whether a new drug is more effective than existing treatments by comparing patient recovery rates. This eliminates the influence of random chance on decision-making by using statistical evidence to support or refute assumptions, thus guiding strategic decisions .