Dataframes & Exploratory Analysis

pandas and Polars, joins, group-bys, time series, and finding the problem in the data before it becomes a problem in the model.

Module 02.3 · Beginner
Free

10h · 8 lessons · 1 challenge

Before this: NumPy & Vectorization

About this module

The first thing you do with a new dataset is find out how it is broken. This module is about that: missingness that is not random, duplicated keys, leaked columns, timestamps in three timezones, categories that appear once.

Covers pandas because that is what the ecosystem runs on, and Polars because that is increasingly what it runs on when the data is large.

After this module you can

8 lessons

Lesson 1
Series, frames, indexes and the pitfalls of each
Kind
Concept
Length
50 min
Lesson 2
Joins, merges and the rows you lost without noticing
Kind
Concept
Length
55 min
Lesson 3
Group-by, aggregation and window functions
Kind
Concept
Length
50 min
Lesson 4
Time series: resampling, lags and timezone bugs
Kind
Concept
Length
50 min
Lesson 5
Leak Detector: find the column that gives it away
Kind
Interactive
Length
40 min
Lesson 6
Full EDA on a messy public dataset
Kind
Lab
Length
100 min
Lesson 7
Polars, and when the rewrite pays for itself
Kind
Concept
Length
40 min
Lesson 8
Checkpoint: what is wrong with this data?
Kind
Checkpoint
Length
25 min

The challenge: Leak Detector

A model scores 0.99 AUC. One column is cheating. Find it across ten datasets of increasing subtlety.

Worth 450 XP.

Related modules

All modules