Module 09 · Core

Data types and safe conversion

Choose a useful data type and identify values that cannot be converted.

Before you start: You can select a column and create a new column.

Python loads from this site on your first run.

Each run starts with fresh data. Download any exported files before another run. Your written answers and code drafts are saved on this device; export your work to keep a backup.

M09C01 · Explore · predict · practice

A number and its text are different

A data type tells pandas what a column contains and which operations make sense. Quotation marks make "20" text. astype() converts values when the requested conversion is possible.

ReadWhat do the quotation marks around the ages tell Python?

PredictWhat will the next-year ages be?

ExplainWhy convert before adding 1?

Read the example

ages = pd.Series(["20", "21", "19"], dtype="string")
result = ages.astype("int64")
print("Before:", ages.dtype)
print("After:", result.dtype)
print("Next year:", (result + 1).tolist())

Verified example output · pandas 2.2.3

Before: string
After: int64
Next year: [21, 22, 20]

Watch for this. Do not assume a column is numeric just because its displayed values look like numbers.

Your turn

Start again with the same text ages. Convert them to numbers and calculate their mean in result.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

ages = pd.Series(["20", "21", "19"], dtype="string")
result = ages.astype("int64")
print("Before:", ages.dtype)
print("After:", result.dtype)
print("Next year:", (result + 1).tolist())

Show a hint

Convert the Series first; then call .mean().

Show one worked solution

ages = pd.Series(["20", "21", "19"], dtype="string")
result = ages.astype("int64").mean()
print(result)

The mean is 20.0. The conversion makes arithmetic meaningful.

Compare your explanations

Read. These values are strings, even though they contain digits.
Predict. 21, 22, and 20.
Explain. We want numeric addition; text values are not numeric ages yet.

One quick check

Which expression requests a numeric type?

What you should be able to explain. Check types before doing arithmetic.

Your own assessment, separate from code checks

M09C02 · Explore · predict · practice

Convert messy numbers and keep the reason

pd.to_numeric() parses values as numbers. With errors="coerce", values it cannot parse become missing. Preserve the source so you can distinguish an invalid label from a value that was already missing.

ReadWhat does errors="coerce" ask pandas to do?

PredictWhich row is marked as invalid text?

ExplainWhy should we keep the original values?

Read the example

source = pd.Series(["88", "absent", None, "72"], dtype="string")
numbers = pd.to_numeric(source, errors="coerce")
invalid = source.notna() & numbers.isna()
result = pd.DataFrame({"Original": source, "Number": numbers, "Invalid_text": invalid})
print(result)

Verified example output · pandas 2.2.3

  Original  Number  Invalid_text
0       88      88         False
1   absent    <NA>          True
2     <NA>    <NA>         False
3       72      72         False

Watch for this. Do not silently turn every conversion failure into zero.

Your turn

Change "absent" to "90" in the source. Rebuild the table and count the missing numbers.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

source = pd.Series(["88", "absent", None, "72"], dtype="string")
numbers = pd.to_numeric(source, errors="coerce")
invalid = source.notna() & numbers.isna()
result = pd.DataFrame({"Original": source, "Number": numbers, "Invalid_text": invalid})
print(result)

Show a hint

Use .isna().sum() after the conversion.

Show one worked solution

source = pd.Series(["88", "90", None, "72"], dtype="string")
numbers = pd.to_numeric(source, errors="coerce")
invalid = source.notna() & numbers.isna()
result = pd.DataFrame({"Original": source, "Number": numbers, "Invalid_text": invalid})
print(result)
print("Missing numbers:", numbers.isna().sum())

Only the original None remains missing; there are no invalid text values.

Compare your explanations

Read. Replace a value that cannot be parsed as a number with a missing value.
Predict. Only row 1, containing "absent". Row 2 was already missing.
Explain. Conversion alone loses the distinction between invalid text and an originally missing value; "absent" may also have a special policy meaning.

One quick check

Does converting "absent" to a missing value mean the score was zero?

What you should be able to explain. Audit conversion failures before deciding how to handle them.

Your own assessment, separate from code checks

M09C03 · Explore · predict · practice

Integers that can be missing

Pandas nullable integer type Int64 can store whole numbers and pd.NA. Its capital I matters. Comparisons can also be missing, so decide how missing comparisons should behave in a filter.

ReadWhat does the capital I in Int64 select?

PredictWhat happens to the missing age when 1 is added?

ExplainWhy is None not an age of zero?

Read the example

ages = pd.Series([20, None, 22], dtype="Int64")
result = ages + 1
print(result)
print("Known ages:", ages.notna().sum())

Verified example output · pandas 2.2.3

0      21
1    <NA>
2      23
dtype: Int64
Known ages: 2

Watch for this. Do not write if pd.NA:; its truth value is ambiguous. Use .isna() or .notna().

Your turn

Start with the same ages and keep only known ages of at least 21. Store the filtered Series in result.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

ages = pd.Series([20, None, 22], dtype="Int64")
result = ages + 1
print(result)
print("Known ages:", ages.notna().sum())

Show a hint

Build ages >= 21; use .fillna(False) on that mask.

Show one worked solution

ages = pd.Series([20, None, 22], dtype="Int64")
mask = (ages >= 21).fillna(False)
result = ages[mask]
print(result)

Only 22 remains. The explicit False says an unknown age does not satisfy this selection rule.

Compare your explanations

Read. Pandas nullable integer type, which supports a missing-value marker.
Predict. It remains missing; pandas does not invent an age.
Explain. It means the age was not recorded, while zero would be an actual number.

One quick check

Which value remains missing after adding 1?

What you should be able to explain. Use nullable types when missing values are legitimate.

Your own assessment, separate from code checks

M09C04 · Explore · predict · practice

Categories and an explicit order

A category stores values drawn from a defined set, such as programs or rating bands. An ordered category has a meaningful sequence that you choose. Sorting follows that sequence instead of alphabetical order.

ReadWhere is the rating order defined?

PredictWhich value appears first after sorting?

ExplainWhy might alphabetical sorting be wrong here?

Read the example

ratings = pd.Series(["High", "Low", "Medium", "Low"], dtype="string")
rating_type = pd.CategoricalDtype(["Low", "Medium", "High"], ordered=True)
result = ratings.astype(rating_type).sort_values()
print(result)
print("Order:", result.cat.categories.tolist())

Verified example output · pandas 2.2.3

1       Low
3       Low
2    Medium
0      High
dtype: category
Categories (3, object): ['Low' < 'Medium' < 'High']
Order: ['Low', 'Medium', 'High']

Watch for this. A value outside a declared category list becomes missing when converted; check the result.

Your turn

Use students(). Make Program an ordered category with IT before AI, then sort by Program and Student_ID. Store the sorted Student_ID and Program columns in the DataFrame result.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

ratings = pd.Series(["High", "Low", "Medium", "Low"], dtype="string")
rating_type = pd.CategoricalDtype(["Low", "Medium", "High"], ordered=True)
result = ratings.astype(rating_type).sort_values()
print(result)
print("Order:", result.cat.categories.tolist())

Show a hint

Create a pd.CategoricalDtype(["IT", "AI"], ordered=True).

Show one worked solution

df = students()
program_type = pd.CategoricalDtype(["IT", "AI"], ordered=True)
df["Program"] = df["Program"].astype(program_type)
result = df.sort_values(["Program", "Student_ID"])[["Student_ID", "Program"]]
print(result)

All six IT students appear before the six AI students; each program is ordered by ID.

Compare your explanations

Read. In the category list ["Low", "Medium", "High"].
Predict. Low, followed by the other Low, then Medium and High.
Explain. Alphabetical sorting puts High first, which does not represent the intended rating order.

One quick check

What does ordered=True add?

What you should be able to explain. Define category meaning and order deliberately.

Your own assessment, separate from code checks

Previous: Create and update columns Next: Missing values and honest cleaning

Further detail: official documentation for this topic.