Scribd
Upload
2 views2 pages

Doc3 Python Data Science Reference

This document provides a quick reference for Python data science, covering core NumPy operations, essential Pandas functionalities, data visualization techniques, and the Scikit-learn workflow. It highlights the importance of using NumPy arrays for efficient computation, Pandas DataFrames for data analysis, and proper data handling practices to avoid common pitfalls in machine learning. Key functions and methods are outlined for each library to facilitate effective data manipulation and analysis.

Uploaded by

Jatin Patel
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
2 views2 pages

Doc3 Python Data Science Reference

This document provides a quick reference for Python data science, covering core NumPy operations, essential Pandas functionalities, data visualization techniques, and the Scikit-learn workflow. It highlights the importance of using NumPy arrays for efficient computation, Pandas DataFrames for data analysis, and proper data handling practices to avoid common pitfalls in machine learning. Key functions and methods are outlined for each library to facilitate effective data manipulation and analysis.

Uploaded by

Jatin Patel
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Python for Data Science: Quick Reference Notes

1. Core NumPy Operations


NumPy arrays are the foundation of most numerical computing in Python. Unlike native Python lists,
NumPy arrays store data in contiguous memory blocks, enabling vectorised operations that are
significantly faster than explicit loops.

• [Link](), [Link](), [Link](), [Link](), [Link]() for creating arrays.

• Broadcasting rules allow operations between arrays of different but compatible shapes without explicit
reshaping.

• Boolean indexing (arr[arr > 0]) is a fast way to filter data conditionally.

2. Pandas Essentials
Pandas DataFrames provide labelled, tabular data structures well suited to exploratory analysis. Common
operations include filtering with boolean masks, grouping with groupby() for aggregate statistics, and
merging datasets with merge() or join().

• [Link]().sum() quickly summarises missing values per column.

• [Link]('category')['value'].mean() computes group-wise averages.

• pd.pivot_table() is useful for building summary tables akin to spreadsheet pivot tables.

• [Link]() lets you run custom row-wise or column-wise functions when vectorised operations are not
straightforward.

3. Data Visualization Notes


Matplotlib remains the base plotting library, while Seaborn provides a higher-level interface with better
default styling for statistical plots. For quick exploratory work, a correlation heatmap ([Link] on
[Link]()) is often the fastest way to spot relationships worth investigating further.

Plotly is preferable when interactivity is needed, such as hovering over data points to inspect exact values,
which is particularly useful in dashboards.

4. Scikit-learn Workflow Pattern


• Split data using train_test_split() before any preprocessing that learns parameters from the data, to
avoid data leakage.

• Fit preprocessing steps (scalers, encoders) only on training data, then transform both train and test
sets.

• Use Pipeline() to chain preprocessing and modelling steps together, which also simplifies
cross-validation.
• GridSearchCV or RandomizedSearchCV for hyperparameter tuning, combined with cross-validation
folds appropriate to the dataset size.

5. Common Pitfalls to Avoid


One frequent mistake is scaling or encoding the entire dataset before splitting into train and test sets,
which leaks information from the test set into training. Another is comparing model performance using only
a single train-test split rather than cross-validation, which can give a misleadingly optimistic or pessimistic
result depending on how the data happened to split.

You might also like