FreshData: public functions across dataset types

Execution-based review of FreshData's public core APIs across representative pandas datasets.

Scope and method

This report executes the nine public core interfaces requested against small, deterministic pandas DataFrames. Results are from the repository's current main branch, not claims copied from the
README.

The scenarios intentionally cover mixed business exports, time-series/sensor readings, protected ID/target/free-text fields, and sparse data. All calls use verbose=False; no source input was
mutated.

Dataset scenarios
Dataset Purpose Notable characteristics

Customer export Business-export cleanup spaced names, string money, sentinels, duplicate, target label

Sensor readings Numeric/time-series safety datetime, missing readings/status, high measurement

Ticket data Protected-field behavior ID, target label, free-text notes, missing values

Sparse data Strategy comparison 70% missing numeric/categorical fields across 40 rows

1. [Link]() - transform data

Customer export

Input:

Customer ID Age Spend ($) Segment Churn

001 34 $1,200.50 Premium 0
002 N/A - Standard 1
002 N/A - Standard 1
003 41 $2,000 None 0

FreshData function and dataset report Page 1

 Output:

customer_id age spend segment churn

001 34 1200.5 Premium 0
002 <NA> NaN Standard 1
003 41 2000.0 None 0
004 <NA> 850.0 Premium 0

Measure Observed result

Shape (5, 5) -> (4, 5)

Missing cells 2 -> 4

Duplicates removed 1

Actions 10

Result dtypes {"customer_id": "object", "age": "Int64", "spend": "float64", "segment": "object", "churn": "int64"}

Sensor readings
Measure Observed result

Shape (20, 4) -> (20, 4)

Missing cells 2 -> 0

Outliers handled 0

Outlier decision preserved 1 outlier(s), 5.0% of values (method=iqr, factor=1.5)

Output columns recorded_at, temperature_c, site_id, status

Protected ticket data

Check Observed result

ticket_id remains missing? True

Target resolved remains missing? True

Free text agent_notes remains missing? True

Warnings target column 'resolved' has 1 missing value(s) (25.0%); rows without a label usually need to be dropped manually

2. [Link]() - inspect without mutation

Dataset Rows x cols Missing cells Duplicate rows Issues

customer export 5x5 2 1 7

sensor readings 20 x 4 2 0 4

protected ticket data 4x4 5 0 5

sparse data 40 x 3 56 0 3

FreshData function and dataset report Page 2

 Profile.to_frame() exposes per-column dtype, missingness, cardinality, sample values, suggested dtype, and issues. The input DataFrames were unchanged after profiling.

3. fd.infer_roles() - classify columns

customer export
column role missing_pct primary_missing_model

Customer ID categorical 0.0 -

Age categorical 20.0 preserve

Churn target 0.0 -

Segment categorical 20.0 preserve

Spend ($) categorical 0.0 -

sensor readings
column role missing_pct primary_missing_model

recorded_at datetime 0.0 -

site_id id 0.0 -

status categorical 5.0 mode

temperature_c numeric 5.0 median

protected ticket data

column role missing_pct primary_missing_model

agent_notes categorical 50.0 preserve

priority numeric 25.0 preserve

resolved categorical 25.0 preserve

ticket_id id 25.0 preserve

4. fd.suggest_plan() - preview decisions

column missing_model missing_confidence outlier_action outlier_model n_outliers

age preserve 0.9 - - 0

segment preserve 0.9 - - 0

spend preserve 0.9 - - 0

The plan is read-only. It shows the primary missing/outlier model and, through alternatives(), the ranked alternatives per column.

FreshData function and dataset report Page 3

 5. fd.compare_plans() - compare proposed strategies
column strategy missing_model outlier_action n_outliers

score balanced preserve - 0

tier balanced preserve - 0

score aggressive preserve - 0

tier aggressive drop - 0

This is also read-only: it compares intended models, not output frames.

6. fd.compare_clean() - compare actual outcomes

strategy rows_befor rows_after cols_before cols_after missing_bef missing_aft missing_del cols_delta duplicates_ outliers_ha columns_dr columns_im duration_se rows_per_s primary_mo
e ore er ta removed ndled opped puted conds econd dels

conservative 40 40 3 3 56 56 0 0 0 0 0 0 0.0157 2555.6 {}

balanced 40 40 3 3 56 56 0 0 0 0 0 0 0.0198 2016.4 {"score":

"preserve",
"tier":
"preserve"}

aggressive 40 40 3 2 56 28 -28 -1 0 0 1 0 0.0206 1939.7 {"score":

"preserve",
"tier": "drop"}

Unlike compare_plans(), this executes each strategy on an isolated copy and summarizes the practical outcome.

7. fd.explain_clean() - explain a cleaning run

ExplainReport attribute Observed value

Rows 5 -> 4

Columns 5 -> 5

Changed-cell estimates {"customer_id": 4, "age": 4, "spend": 4, "segment": 4, "churn": 4}

Pipeline steps column_names, normalize_sentinels, strip_whitespace, fix_dtypes, drop_duplicates, missing

Narratives age: preserved 2 missing value(s) (role=numeric, missing=50.0%, strategy='balanced') | segment: preserved 1 missing
value(s) (role=categorical, missing=25.0%, strategy='balanced') | spend: preserved 1 missing value(s) (role=numeric,
missing=25.0%, strategy='balanced')

It adds a before/after explanation and inferred roles on top of the ordinary CleanReport.

8. [Link] - reuse one policy

FreshData function and dataset report Page 4

 Check Observed result

Configured strategy balanced

Output equals direct [Link]() result True

Latest report retained as report_ True

Actions in retained report 10

Use this object when multiple frames must follow the same configuration.

9. [Link] - validate and make behavior reproducible

Property Observed value

Immutable dataclass True

Default strategy balanced

Default outlier action auto

Resolved IQR factor 1.5

Invalid configuration response strategy must be one of ('conservative', 'balanced', 'aggressive', 'auto'), got 'unsafe'

The config controls naming, strings, dtype inference, duplicates, missingness, outliers, memory settings, and safety guards. Invalid values fail early rather than being silently ignored.

Takeaways
- Start with profile() and infer_roles() when learning a dataset; neither changes it.
- Use suggest_plan() before automated changes when missingness or outlier handling is sensitive.
- balanced is the appropriate first strategy for most real datasets; the sparse-data comparison shows why aggressive changes should be reviewed.
- Provide target_column and id_columns explicitly whenever you know them. That makes the safety policy unambiguous.
- Keep CleanReport or ExplainReport with pipeline outputs if auditability matters.

FreshData function and dataset report Page 5