DataIntermediate

Profile and clean a messy dataset

A cleaning plan that documents every decision so the analysis is reproducible.

Works well with

When to use it

When the CSV has dates in three formats and you do not know how many records are real.

Copy Prompt

Copy the prompt and replace the placeholders in braces.

Act as a data engineer. Cleaning must be reproducible and auditable.

Dataset description: {{description}}
Sample rows: {{sample}}
What I am going to use it for: {{goal}}

Deliver:

1. PROFILING: what to check column by column (types, nulls, duplicates, ranges, inconsistent categories).
2. LIKELY PROBLEMS ordered by impact on my goal, not by technical interest.
3. CLEANING RULES: for each problem, what to do and what NOT to do (e.g. do not impute the mean in a column with 40% nulls).
4. Python/pandas CODE that implements the rules and leaves a log of how many rows it changed.
5. VALIDATION: 5 checks after cleaning.
6. WHAT TO DOCUMENT so someone else can repeat it in 6 months.

Rules: never delete records without justifying and counting them. Distinguish null from zero and from empty. If a cleaning decision could bias the analysis, say so explicitly.

Variables to fill in

Variable	What it means	Example
`description`	What the dataset contains	3 years of sales exported from 2 systems
`sample`	10-20 sample rows	(paste CSV)
`goal`	What you will use it for	Calculate average order value per channel

Tips

The log of modified rows is what saves you when someone questions the number.
Never impute without looking at the distribution first.
Keep the original dataset intact and write on a copy.

Editorial template. Test it with real data before using it in production.