Introduction to Pandas¶

What is Pandas?¶

Pandas is the standard Python library for working with tabular data — anything that fits in a spreadsheet:

Customer records (name, age, city, plan)
Sales over time
Sensor readings
Survey results

Pandas is built on top of NumPy, adds labels (named columns, indexed rows), and provides hundreds of functions for cleaning, transforming, and analyzing data.

The two main objects¶

Object	What it is	Like
`Series`	A 1D labeled array	One column
`DataFrame`	A 2D table	A whole spreadsheet — rows + columns

import pandas as pd

# A Series (one column)
ages = pd.Series([25, 30, 35], name="age")
print(ages)
print(type(ages).__name__)

import pandas as pd

# A DataFrame (a table)
df = pd.DataFrame({
    "name": ["Alice", "Bob", "Carol"],
    "age":  [25, 30, 35],
    "city": ["Mumbai", "Delhi", "Pune"],
})
print(df)
print()
print("shape:", df.shape)
print("columns:", df.columns.tolist())

Why Pandas?¶

Compared to working with raw Python lists / dicts:

Task	Plain Python	Pandas
Read a CSV	many lines	`pd.read_csv("file.csv")`
Compute average per city	nested loops	`df.groupby("city")["age"].mean()`
Filter rows	list comprehension	`df[df.age > 30]`
Merge two datasets	manual joins	`df1.merge(df2, on="id")`
Pivot tables	nightmare	`df.pivot_table(...)`
Time series	hard	built-in

Pandas takes hours of tedious data wrangling and turns it into a few clean lines.

Why Pandas matters¶

Pandas is the starting point of every data project in Python:

Load data → CLEAN with Pandas → analyze → model (sklearn) → present

Knowing Pandas well is what makes you productive at: - Data science - Machine learning (Pandas → sklearn / PyTorch / XGBoost) - Quantitative finance - Business analytics - Scientific research

A 30-second taste¶

import pandas as pd

# Create a small dataset
df = pd.DataFrame({
    "name":   ["Alice", "Bob", "Carol", "Dave", "Eve"],
    "age":    [25, 32, 47, 51, 23],
    "city":   ["Mumbai", "Delhi", "Mumbai", "Pune", "Delhi"],
    "salary": [50_000, 65_000, 80_000, 90_000, 45_000],
})

# Show first rows
print(df.head())
print()

# Summary statistics
print(df.describe())
print()

# Average salary per city
print(df.groupby("city")["salary"].mean())
print()

# People over 30
print(df[df.age > 30])

All of that in 5 minutes of code. Try the same with plain lists — you'll spend an afternoon.

Pandas vs NumPy — when to use which¶

Task	NumPy	Pandas
Pure numeric arrays / matrices	✅	overkill
Heterogeneous columns (numbers + strings + dates)	hard	✅
Named columns	no	✅
Time series with date index	hard	✅
Image / audio / scientific tensors	✅	rarely
Deep learning tensors	NumPy → PyTorch / TF	not directly
Data cleaning + analysis	painful	✅
Reading CSV / Excel / SQL	manual	one line

Often you use both: Pandas for the high level, NumPy under the hood when you need raw speed.

What you'll learn in this tutorial¶

#	Chapter
2	Creating Series and DataFrames
3	Reading & writing files (CSV, JSON, Excel, SQL)
4	Inspecting data — `head`, `info`, `describe`
5	Selecting columns and rows (`loc`, `iloc`)
6	Filtering with conditions
7	Adding, removing, modifying columns
8	Missing data — `isna`, `fillna`, `dropna`
9	Sorting and ranking
10	GroupBy — split-apply-combine
11	Merge, join, concatenate
12	Reshape — pivot, melt, stack
13	Time series — dates, resampling, rolling
14	Real-world examples — end-to-end cleaning + analysis

Prerequisites¶

Python basics — dicts, lists, loops.
A passing familiarity with NumPy helps but isn't required.

A note on the runnable code¶

Every Python block has a ▶ Run button. Click to execute in your browser — first run downloads Python + Pandas (~30 MB, one-time, takes ~20s). After that, everything is instant.

Practice¶

What does this print?

Expected: (3, 3)

import pandas as pd
df = pd.DataFrame({"a": [1,2,3], "b": [4,5,6], "c": [7,8,9]})
print(df.shape)

Create a DataFrame from two equal-length columns

Expected: name age

import pandas as pd
df = pd.DataFrame({"name": ["Alice", "Bob"], "age": [25, 30, 35]})   # bug: lengths differ
print(df)

Quiz — Quick check¶

What you remember

Q1. What's the difference between a Series and a DataFrame?

Series is faster
Series is a 1D labeled array (one column); DataFrame is a 2D table with multiple columns
Series can only hold numbers
They're identical

Why: A DataFrame is a collection of Series sharing the same index. Selecting a single column from a DataFrame (df["col"]) gives you a Series.

Q2. Which is faster for filtering 1 million rows?

Python list comprehension
Pandas boolean filter (df[df.col > 5])
A for loop
df.apply(lambda r: r.col > 5, axis=1)

Why: Boolean filters dispatch to vectorized NumPy operations — much faster than Python loops or .apply(). Use vectorized ops first; reach for .apply() only when you must.

Q3. Pandas is built on top of which library?

scikit-learn
NumPy
TensorFlow
SciPy

Why: A Pandas DataFrame is essentially a 2D NumPy array with labeled rows/columns. That's why NumPy operations carry over and why Pandas is fast on numeric data.

Common doubts¶

When should I use Pandas vs raw NumPy?

Use Pandas for tabular data with mixed types, named columns, time-indexed data, or anything that looks like a spreadsheet. Use NumPy for pure numeric arrays, especially high-dimensional ones (images, tensors). They work together — Pandas uses NumPy under the hood.

Is Pandas slow on big data?

"Big" is relative. For ≤ 1–10 million rows, Pandas is fine. For larger data look at Polars (faster, similar API), DuckDB (SQL on Parquet/CSV), or Dask/Modin (out-of-core Pandas). The migration is usually small because the APIs are similar.

Why does Pandas show NaN for missing values?

NaN (Not a Number) is the float standard for "missing". Pandas adopts it because numerical columns must store something — None doesn't fit in a float column. Modern Pandas (1.0+) also has pd.NA for non-float columns, but NaN is what you'll see most often.

→ Creating DataFrames