pbPassingBI
/
Working with DataFrames beginner 6 min

Loading data

read_csv and read_excel, and the arguments that save you an hour.

What you'll be able to do
  • Load CSV and Excel files
  • Handle separators, encodings and dates
  • Read a specific Excel sheet

Reading a CSV

import pandas as pd

df = pd.read_csv('sales.csv')

That covers the easy case. The arguments below cover most of the rest.

The arguments worth knowing

ArgumentUse
sep=';'European CSVs are often semicolon-separated
encoding='latin-1'When UTF-8 raises a decode error
parse_dates=['order_date']Read dates as dates, not text
dtype={'zip': str}Stop leading zeros being stripped
usecols=['a','b']Read only what you need
nrows=1000Sample a large file first
thousands=','Parse 1,250 as a number
na_values=['N/A','-']Treat placeholders as missing

The zip code problem

Leading zeros disappear

A zip code column of 07030 is read as the integer 7030. So are product codes, account numbers and anything else that looks numeric but is not.

df = pd.read_csv('data.csv', dtype={'zip': str, 'account_id': str})

If a column is an identifier rather than a quantity, read it as a string. You will not be doing arithmetic on it anyway.

Excel

df = pd.read_excel('report.xlsx', sheet_name='Q1')
all_sheets = pd.read_excel('report.xlsx', sheet_name=None)   # dict of DataFrames
df = pd.read_excel('report.xlsx', skiprows=3)                # skip title rows

sheet_name=None returning a dictionary keyed by sheet name is genuinely useful for workbooks with one sheet per month.

Other sources

pd.read_json('data.json')
pd.read_parquet('data.parquet')
pd.read_sql('SELECT * FROM orders', con=engine)
pd.read_clipboard()      # quick paste from Excel
Key points
  • Read identifier columns as str to preserve leading zeros
  • parse_dates at load time avoids converting later
  • sheet_name=None reads every Excel sheet into a dictionary
Check yourself