How do you find and remove duplicate records?

df.duplicated().sum() counts fully identical rows; duplicated(subset=['email']) counts duplicates on a key.

df.drop_duplicates(subset=['email']) removes them, with keep controlling whether the first, last or none survive.

The part that matters in practice is deciding which copy to keep. Sort first so the choice is deliberate:

df.sort_values('updated_at').drop_duplicates(subset=['email'], keep='last')

And normalise text keys — .str.strip().str.lower() — before deduplicating, or near-duplicates that differ only in case survive.