Class 12 Data Handling with Pandas: Complete Concepts and Practice

Big picture

Pandas is the most widely used Python library for working with tabular data. It gives two main data structures: the Series, a one-dimensional labelled array (like one column), and the DataFrame, a two-dimensional labelled table (like a spreadsheet or SQL table).

The Informatics Practices board paper always has questions on creating Series and DataFrames, selecting data with loc and iloc, filtering with conditions, adding and deleting columns, and predicting outputs. Every example below was run to confirm the output.

Learning goals

Create a Series from a list, dictionary, scalar and NumPy array, with custom indexes.
Access Series elements by label and position; slice; use head, tail and vector operations.
Explain index alignment and NaN in arithmetic between Series.
Create a DataFrame from dictionaries and lists, and read its attributes.
Select rows and columns with [], loc, iloc and boolean conditions.
Add, rename and delete rows and columns; sort and iterate.
Calculate statistics and import or export CSV files.

Core ideas

1. Series

import pandas as pd

s = pd.Series([10, 20, 30, 40], index=['a', 'b', 'c', 'd'])
print(s['b'])        # 20  (by label)
print(s.iloc[2])     # 30  (by position)
print(s.size)        # 4

Other ways to create a Series:

From a dictionary: keys become the index. pd.Series({'x': 5, 'y': 7}) has index ['x', 'y'].
From a scalar: pd.Series(0, index=['a', 'b', 'c']) repeats 0 for each label.
From a NumPy array: pd.Series(np.array([1, 2, 3])).
Without an index, the default index is 0, 1, 2 …

Series attributes: index, values, size, dtype, name, empty, hasnans.

2. Selecting, slicing and operations on a Series

print(s[s > 15])          # b 20, c 30, d 40 (boolean indexing)
print(s.head(2))          # first 2 elements: a 10, b 20
print(s.tail(2))          # last 2 elements: c 30, d 40
print(s * 2)              # vector operation: 20, 40, 60, 80
print(s['b':'c'])         # label slice INCLUDES the end label: 20, 30
print(s.iloc[1:3])        # position slice EXCLUDES the end position: 20, 30

head() and tail() show 5 elements by default.

Index alignment. Arithmetic between two Series matches values by index label. A label present in only one Series gives NaN (not a number).

a = pd.Series([1, 2, 3], index=['p', 'q', 'r'])
b = pd.Series([10, 20, 30], index=['q', 'r', 's'])
print(a + b)
# p     NaN
# q    12.0
# r    23.0
# s     NaN

Use a.add(b, fill_value=0) to treat missing values as 0 instead.

3. Creating a DataFrame

df = pd.DataFrame({'Name': ['Asha', 'Ravi', 'Neha', 'Arun'],
                   'Marks': [88, 72, 91, 65],
                   'City': ['Pune', 'Delhi', 'Pune', 'Agra']})
print(df.shape)              # (4, 3)
print(df.columns.tolist())   # ['Name', 'Marks', 'City']

Other ways: from a list of dictionaries (each dictionary is a row), from a list of lists with columns=[...], from a dictionary of Series, or from a CSV file.

DataFrame attributes: index, columns, shape (rows, columns), size (total elements), ndim (2), dtypes, values, T (transpose), empty.

4. Selecting data

Method	Selects by	Example	Result
`df['Marks']` or `df.Marks`	Column name		The Marks column
`df[['Name', 'City']]`	Several columns		Two columns
`df.loc[row_label, col_label]`	Labels	`df.loc[1, 'Name']`	Ravi
`df.iloc[row_pos, col_pos]`	Positions	`df.iloc[2, 1]`	91
`df.loc[1:2, 'Name']`	Label slice (end included)		Ravi, Neha
`df.iloc[1:2, 0]`	Position slice (end excluded)		Ravi
Boolean indexing	Condition	`df.loc[df['Marks'] > 80, 'Name']`	Asha, Neha

Combining conditions: use & (and), | (or) and brackets: df[(df.Marks > 70) & (df.City == 'Pune')].

Example: df[df.City == 'Pune'].Marks.sum() gives 88 + 91 = 179.

5. Adding, renaming and deleting

df['Grade'] = ['A', 'B', 'A', 'C']          # add a column; shape becomes (4, 4)
df.loc[4] = ['Isha', 79, 'Agra', 'B']       # add a row with label 4
df = df.rename(columns={'Marks': 'Score'})  # rename a column
d2 = df.drop('Grade', axis=1)               # delete a column
d3 = df.drop(0)                             # delete the row with label 0 (axis=0 is the default)

axis=0 refers to rows and axis=1 to columns.
drop() and rename() return a new DataFrame unless the result is assigned back (many textbooks write inplace=True, which changes the original directly).

6. Sorting, iteration and statistics

print(df.sort_values('Score', ascending=False).Name.tolist()[:2])   # ['Neha', 'Asha']

for i, row in df.head(2).iterrows():     # iterate row by row
    print(i, row['Name'])                # 0 Asha, then 1 Ravi

iterrows() gives (index, row) pairs; items() gives (column name, column) pairs.
Statistics: mean(), median(), max(), min(), sum(), count(), mode(), std(), var(), quantile(). For the original marks, df['Marks'].mean() is 79.0 and max() is 91.
describe() gives count, mean, std, min, quartiles and max for numeric columns.

7. Importing and exporting CSV files

df = pd.read_csv('students.csv')                  # first row is used as column names
df = pd.read_csv('students.csv', header=None)     # no header row in the file
df.to_csv('result.csv', index=False)              # write without the index column

Missing data: isnull() or isna() finds NaN values; dropna() removes rows with NaN; fillna(value) replaces NaN.

A reliable routine for Pandas questions

Decide whether you are selecting by label (loc, includes the end of a slice) or position (iloc, excludes the end).
For conditions, put the condition inside df[...] or df.loc[...], using & and | with brackets.
Remember axis=1 for columns and axis=0 for rows.
For arithmetic between Series, match labels and expect NaN for unmatched ones.
Check whether a method changes the original or returns a new object.

Connections that matter

Data visualisation plots Series and DataFrame data with Matplotlib.
Advanced SQL queries: SELECT, WHERE and GROUP BY have Pandas equivalents (selection, boolean indexing, groupby()).
Data communication project: projects use Pandas to clean and analyse data.

Common mistakes

Thinking `loc` slices exclude the end label. loc includes it; iloc excludes it.
Using `and`/`or` instead of `&`/`|` in DataFrame conditions.
Forgetting `axis=1` when dropping a column.
Expecting a missing label to give 0 in Series arithmetic. It gives NaN.
Confusing `shape` and `size`. shape is (rows, columns); size is the total number of elements.

Revision checklist

I can create Series and DataFrames in every standard way.
I can select by label and position, slice, and filter with conditions.
I can explain index alignment and NaN.
I can add, rename and delete rows and columns.
I can sort, iterate and calculate statistics.
I can read and write CSV files and handle missing values.