Module 22 · Extension

Explore richer structures

Use multiple index levels, list-valued columns, and categorical encodings.

Before you start: Understand indexes, reshaping, and basic data types.

Python loads from this site on your first run.

Each run starts with fresh data. Download any exported files before another run. Your written answers and code drafts are saved on this device; export your work to keep a backup.

M22C01 · Explore · predict · practice

Create a MultiIndex

A MultiIndex stores more than one label per row. Here Campus and Student_ID jointly locate a student. sort_index arranges the levels for predictable selection.

Read the example

df = students().set_index(["Campus", "Student_ID"]).sort_index()
result = df.loc[("RAK", "S003"), ["Name", "Score"]]
print(result)
Verified example output · pandas 2.2.3
Name     Chloe
Score       91
Name: (RAK, S003), dtype: object

Watch for this. A tuple used as a MultiIndex key is different from a list of separate row keys.

Your turn

Use xs to select all Dubai students at the Campus level and print Name and Score.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Show a hint

Call xs("Dubai", level="Campus") on the indexed table.

Show one worked solution
df = students().set_index(["Campus", "Student_ID"]).sort_index()
result = df.xs("Dubai", level="Campus")[["Name", "Score"]]
print(result)

The selection contains six Dubai students; xs drops the selected level by default.

Compare your explanations
  1. Read. A row with Campus RAK and Student_ID S003.
  2. Predict. Chloe and 91.
  3. Explain. Campus alone is shared by several students; the second level identifies the individual record.

One quick check

What is a MultiIndex?

What you should be able to explain. Name index levels so selections are understandable.

M22C02 · Explore · predict · practice

Stack and unstack

stack moves column labels into an additional row-index level. unstack moves an index level back into columns. The explicit future_stack=True selects the modern stack implementation on pandas 2.2 and 3.x.

Read the example

wide = students().head(2).set_index("Student_ID")[["Score", "Attendance"]]
result = wide.stack(future_stack=True)
print(result)
Verified example output · pandas 2.2.3
Student_ID            
S001        Score         88
            Attendance    95
S002        Score         72
            Attendance    85
dtype: int64

Watch for this. Missing combinations can appear when reshaping sparse data; inspect the resulting shape.

Your turn

Stack the first three students and unstack the last level back into a DataFrame. Check that it equals the original wide table.

For the code check, keep these result variable names: result, wide. Different valid approaches are welcome.

Show a hint

Use .unstack() on the stacked Series, then .equals(wide).

Show one worked solution
wide = students().head(3).set_index("Student_ID")[["Score", "Attendance"]]
stacked = wide.stack(future_stack=True)
result = stacked.unstack().reindex(columns=wide.columns)
print(result)
print("Values restored:", result.equals(wide))

The explicit column reindex restores the original column order for the equality check.

Compare your explanations
  1. Read. Two: Student_ID and the original measure column label.
  2. Predict. 4, from two students and two measures.
  3. Explain. No. Their labels moved; the original values remain.

One quick check

Which operation returns the last index level to columns?

What you should be able to explain. Stack and unstack move labels between axes.

M22C03 · Explore · predict · practice

Expand lists with explode

Sometimes one cell contains several items, such as a list of skills. explode creates a separate row for each item and repeats the other columns. This changes the unit of analysis from one row per student to one row per student-skill.

Read the example

df = pd.DataFrame({"Student_ID": ["S001", "S002"], "Skills": [["Python", "SQL"], ["Excel"]]})
result = df.explode("Skills", ignore_index=True)
print(result)
Verified example output · pandas 2.2.3
  Student_ID  Skills
0       S001  Python
1       S001     SQL
2       S002   Excel

Watch for this. An empty list becomes a row with a missing item; a string that looks like a list is still a string.

Your turn

Give S002 a second skill, SQL. Explode the table, then store the count of student-skill rows for each skill in result as a value_counts() Series.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Show a hint

After explode, use the Skills column value_counts().

Show one worked solution
df = pd.DataFrame({"Student_ID": ["S001", "S002"], "Skills": [["Python", "SQL"], ["Excel", "SQL"]]})
long = df.explode("Skills", ignore_index=True)
result = long["Skills"].value_counts()
print(result)

SQL appears twice; Python and Excel appear once each.

Compare your explanations
  1. Read. A Python list.
  2. Predict. 3: two skills for S001 and one skill for S002.
  3. Explain. A student with multiple skills appears in multiple rows.

One quick check

After explode, one row represents what?

What you should be able to explain. Check what a row represents after changing the layout.

M22C04 · Explore · predict · practice

Order and encode categories

Categorical values can follow a chosen order instead of alphabetical order. get_dummies makes one indicator column per category. These indicator values describe membership, not a numerical ranking.

Read the example

df = students().head(4).copy()
df["Program"] = pd.Categorical(df["Program"], categories=["IT", "AI"], ordered=True)
ordered = df.sort_values("Program", kind="stable")
result = pd.get_dummies(ordered["Program"], prefix="Program", dtype=int)
print(ordered[["Student_ID", "Program"]])
print(result)
Verified example output · pandas 2.2.3
  Student_ID Program
1       S002      IT
3       S004      IT
0       S001      AI
2       S003      AI
   Program_IT  Program_AI
1           1           0
3           1           0
0           0           1
2           0           1

Watch for this. Values outside a specified category list become missing; validate allowed categories first.

Your turn

Change the category order to AI then IT and print the sorted student IDs and dummy columns.

For the code check, keep these result variable names: ordered, result. Different valid approaches are welcome.

Show a hint

Reverse the categories list before sorting.

Show one worked solution
df = students().head(4).copy()
df["Program"] = pd.Categorical(df["Program"], categories=["AI", "IT"], ordered=True)
ordered = df.sort_values("Program", kind="stable")
result = pd.get_dummies(ordered["Program"], prefix="Program", dtype=int)
print(ordered[["Student_ID", "Program"]])
print(result)

AI students now appear first; each row still has one indicator value equal to 1.

Compare your explanations
  1. Read. IT, because it is listed first in the ordered categories.
  2. Predict. 0 and 1 respectively.
  3. Explain. No. It only marks whether that row belongs to the AI category.

One quick check

What does get_dummies encode?

What you should be able to explain. Category order and indicator encoding are separate choices.

Further detail: official documentation for this topic.