Module 12 · Core

Clean and search text

Standardize labels and extract useful pieces from text columns.

Before you start: You can select columns and filter with Boolean masks.

Python loads from this site on your first run.

Each run starts with fresh data. Download any exported files before another run. Your written answers and code drafts are saved on this device; export your work to keep a backup.

M12C01 · Explore · predict · practice

Clean spaces and preserve meaningful labels

The .str accessor applies text operations to each entry in a Series. strip() removes spaces at the beginning and end. A label map can then standardize spelling while preserving the exact labels you want.

ReadWhat does .str.strip() remove?

PredictWhat will S001 Name become?

ExplainWill strip remove the space inside a two-word name?

Read the example

df = raw_students()
df["Name"] = df["Name"].astype("string").str.strip()
result = df.loc[df["Student_ID"] == "S001", ["Student_ID", "Name"]]
print(result)

Verified example output · pandas 2.2.3

  Student_ID   Name
0       S001  Amira

Watch for this. An unrecognized value passed through a dictionary map becomes missing; inspect unmatched labels.

Your turn

Standardize all Campus labels as exactly RAK or Dubai. Handle case and surrounding spaces before mapping. Store Student_ID and the cleaned Campus for every source row in result.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

df = raw_students()
df["Name"] = df["Name"].astype("string").str.strip()
result = df.loc[df["Student_ID"] == "S001", ["Student_ID", "Name"]]
print(result)

Show a hint

Use .str.strip().str.lower() followed by .map({"rak": "RAK", "dubai": "Dubai"}).

Show one worked solution

df = raw_students()
campus_text = df["Campus"].astype("string").str.strip().str.lower()
df["Campus"] = campus_text.map({"rak": "RAK", "dubai": "Dubai"})
result = df[["Student_ID", "Campus"]]
print(result)

S002 changes from dubai to Dubai. The abbreviation RAK remains uppercase.

Compare your explanations

Read. Leading and trailing whitespace from each text value.
Predict. Amira, without the surrounding spaces.
Explain. No. Internal spaces are preserved.

One quick check

Would .str.title() alone preserve RAK exactly?

What you should be able to explain. Standardize categories without destroying their meaning.

Your own assessment, separate from code checks

M12C02 · Explore · predict · practice

Literal text and regular expressions

str.contains() checks whether text contains a pattern. Use regex=False when you want the characters literally. With a regular expression, characters such as a dot can have special meanings.

ReadWhy does the code set regex=False?

PredictWhich two filenames are selected?

ExplainWhat does na=False do?

Read the example

files = pd.Series(["report.csv", "reportXcsv", "notes.csv", None, "csv.backup"], dtype="string")
mask = files.str.contains(".csv", regex=False, na=False)
result = files[mask]
print(result)

Verified example output · pandas 2.2.3

0    report.csv
2     notes.csv
dtype: string

Watch for this. Always be clear whether a pattern is literal text or a regular expression.

Your turn

Select all nonmissing filenames that begin with report. Use a regular expression and store the result.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

files = pd.Series(["report.csv", "reportXcsv", "notes.csv", None, "csv.backup"], dtype="string")
mask = files.str.contains(".csv", regex=False, na=False)
result = files[mask]
print(result)

Show a hint

The ^ symbol anchors a regular expression to the start of the text.

Show one worked solution

files = pd.Series(["report.csv", "reportXcsv", "notes.csv", None, "csv.backup"], dtype="string")
mask = files.str.contains(r"^report", regex=True, na=False)
result = files[mask]
print(result)

Both report.csv and reportXcsv begin with report. Their extensions do not matter for this question.

Compare your explanations

Read. The dot should be an actual dot, not a pattern matching any character.
Predict. report.csv and notes.csv.
Explain. It treats a missing text entry as not matching the search.

One quick check

Which argument makes a search literal?

What you should be able to explain. Choose text-search semantics deliberately.

Your own assessment, separate from code checks

M12C03 · Explore · predict · practice

Split a value into separate columns

str.split() separates text at a delimiter. expand=True creates separate columns rather than a list in each cell. Limiting splits can protect the rest of a string when a delimiter appears repeatedly.

ReadWhat character divides the email into two parts?

PredictWhat will the first Username be?

ExplainWhy use expand=True?

Read the example

emails = pd.Series(["amira@example.edu", "bilal@example.edu", None], dtype="string")
result = emails.str.split("@", n=1, expand=True, regex=False)
result.columns = ["Username", "Domain"]
print(result)

Verified example output · pandas 2.2.3

  Username       Domain
0    amira  example.edu
1    bilal  example.edu
2     <NA>         <NA>

Watch for this. This is a string-processing exercise, not full validation of every legal email address.

Your turn

Use the same emails. Extract Domain as a Series, then store the frequency of each known domain in result as a value_counts() Series.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

emails = pd.Series(["amira@example.edu", "bilal@example.edu", None], dtype="string")
result = emails.str.split("@", n=1, expand=True, regex=False)
result.columns = ["Username", "Domain"]
print(result)

Show a hint

Select the second split piece with .str[1], then call .value_counts().

Show one worked solution

emails = pd.Series(["amira@example.edu", "bilal@example.edu", None], dtype="string")
domains = emails.str.split("@", n=1, regex=False).str[1]
result = domains.value_counts()
print(result)

example.edu has two known occurrences. Missing domains are excluded by value_counts() by default.

Compare your explanations

Read. The @ character.
Predict. amira.
Explain. We want two table columns, so each piece can be selected and analyzed separately.

One quick check

What object does expand=True produce here?

What you should be able to explain. Split text into fields before summarizing its parts.

Your own assessment, separate from code checks

M12C04 · Explore · predict · practice

Extract only values that match a structure

str.extract() uses capture groups to pull parts out of a matching pattern. Named groups become useful column names. A nonmatching string produces missing fields, which you should inspect.

ReadWhat do Program and Number name in the pattern?

PredictWhat happens to the value bad?

ExplainWhy are ^ and $ useful here?

Read the example

codes = pd.Series(["AI-101", "IT-205", None, "bad"], dtype="string")
result = codes.str.extract(r"^(?P<Program>[A-Z]+)-(?P<Number>\d+)$")
print(result)

Verified example output · pandas 2.2.3

  Program Number
0      AI    101
1      IT    205
2    <NA>   <NA>
3    <NA>   <NA>

Watch for this. A successful pattern match checks structure, not whether the code exists in an official registry.

Your turn

Use the same codes. Extract the fields and convert Number to nullable Int64.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

codes = pd.Series(["AI-101", "IT-205", None, "bad"], dtype="string")
result = codes.str.extract(r"^(?P<Program>[A-Z]+)-(?P<Number>\d+)$")
print(result)

Show a hint

Pass the extracted Number column through pd.to_numeric(..., errors="coerce").

Show one worked solution

codes = pd.Series(["AI-101", "IT-205", None, "bad"], dtype="string")
result = codes.str.extract(r"^(?P<Program>[A-Z]+)-(?P<Number>\d+)$")
result["Number"] = pd.to_numeric(result["Number"], errors="coerce").astype("Int64")
print(result)

The valid numeric codes are 101 and 205. The missing and nonmatching source entries remain missing.

Compare your explanations

Read. The two captured fields, which become output columns.
Predict. Both extracted fields are missing because the whole pattern does not match.
Explain. They require the full string to match the structure, rather than accepting a matching piece inside other text.

One quick check

Does extract automatically turn captured digits into integers?

What you should be able to explain. Extract structured pieces and then assign appropriate types.

Your own assessment, separate from code checks

Previous: Duplicates and data quality Next: Dates, times, and durations

Further detail: official documentation for this topic.