Module 15 · Core

Statistics you can explain

Summarize center, spread, completeness, and association with clear interpretations.

Before you start: You can select numeric columns and identify missing values.

Python loads from this site on your first run.

Each run starts with fresh data. Download any exported files before another run. Your written answers and code drafts are saved on this device; export your work to keep a backup.

M15C01 · Explore · predict · practice

Mean and median respond differently

The mean uses every numeric value in an arithmetic average. The median is the middle value after sorting. Comparing them can help you notice how an unusually large or small observation affects a summary.

ReadWhat calculation produces the mean here?

PredictWhat is the median?

ExplainDoes a mean equal to the median prove the data are correct?

Read the example

costs = pd.Series([10, 20, 30, 40, 50])  # Fictional costs in AED
result = pd.Series({"Mean_AED": costs.mean(), "Median_AED": costs.median()})
print(result)

Verified example output · pandas 2.2.3

Mean_AED      30.0
Median_AED    30.0
dtype: float64

Watch for this. An extreme value is not automatically a mistake; investigate its source and meaning.

Your turn

Change the last cost from 50 to 500. Recalculate mean and median in result.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

costs = pd.Series([10, 20, 30, 40, 50])  # Fictional costs in AED
result = pd.Series({"Mean_AED": costs.mean(), "Median_AED": costs.median()})
print(result)

Show a hint

Recreate the Series with the changed final value.

Show one worked solution

costs = pd.Series([10, 20, 30, 40, 500])
result = pd.Series({"Mean_AED": costs.mean(), "Median_AED": costs.median()})
print(result)

The mean rises to AED 120, while the median stays AED 30. The large value has a stronger effect on the mean.

Compare your explanations

Read. (10 + 20 + 30 + 40 + 50) / 5 = 30.
Predict. 30, the middle value of the five sorted costs.
Explain. No. It only describes this set of numbers; validation is a separate task.

One quick check

Which summary stays 30 after this change?

What you should be able to explain. Explain which summary fits the question and distribution.

Your own assessment, separate from code checks

M15C02 · Explore · predict · practice

Spread and the standard-deviation convention

Standard deviation measures spread around the mean in the same units as the values. pandas uses sample standard deviation by default, with ddof=1. ddof=0 uses the population-style denominator; choose based on what the data represent.

ReadWhat does ddof change?

PredictWhich standard deviation is larger for these data?

ExplainWould changing hours to minutes leave the standard deviation numerically unchanged?

Read the example

hours = pd.Series([2.0, 4.0, 6.0])
result = pd.Series({"Mean_hours": hours.mean(), "Sample_std_hours": hours.std(ddof=1), "Population_std_hours": hours.std(ddof=0)})
print(result.round(4))

Verified example output · pandas 2.2.3

Mean_hours              4.000
Sample_std_hours        2.000
Population_std_hours    1.633
dtype: float64

Watch for this. With only one known value, sample std is missing because n - 1 is zero.

Your turn

Replace the hours with [4.0, 4.0, 4.0]. Recalculate both standard deviations.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

hours = pd.Series([2.0, 4.0, 6.0])
result = pd.Series({"Mean_hours": hours.mean(), "Sample_std_hours": hours.std(ddof=1), "Population_std_hours": hours.std(ddof=0)})
print(result.round(4))

Show a hint

A set of identical values has no spread around its mean.

Show one worked solution

hours = pd.Series([4.0, 4.0, 4.0])
result = pd.Series({"Sample_std_hours": hours.std(ddof=1), "Population_std_hours": hours.std(ddof=0)})
print(result)

Both standard deviations are zero because every value equals the mean.

Compare your explanations

Read. It changes the denominator from n to n minus ddof in the variance calculation.
Predict. The sample value, 2.0, is larger than the population-style value, about 1.633.
Explain. No. The standard deviation would also be multiplied by 60 because it has the same units.

One quick check

Which ddof does pandas std() use by default?

What you should be able to explain. State the spread measure and its convention.

Your own assessment, separate from code checks

M15C03 · Explore · predict · practice

Describe, count, and quantiles

describe() summarizes a numeric Series with count, mean, spread, and selected percentiles. Count excludes missing values; size counts all entries. A quantile describes a position in the distribution, and its value may be interpolated between observations.

ReadDoes describe count the missing score as an observed score?

PredictWhat is the 50% value?

ExplainWhy does a reported average need a known-value count?

Read the example

scores = pd.Series([60.0, 70.0, np.nan, 80.0, 90.0])
result = scores.describe()
print(result)
print("Rows:", scores.size, "Known:", scores.count())

Verified example output · pandas 2.2.3

count     4.000000
mean     75.000000
std      12.909944
min      60.000000
25%      67.500000
50%      75.000000
75%      82.500000
max      90.000000
dtype: float64
Rows: 5 Known: 4

Watch for this. Describe is a starting point. It does not check whether IDs, units, ranges, or collection procedures are valid.

Your turn

For the same scores, report the 25th, 50th, and 75th percentiles explicitly using quantile.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

scores = pd.Series([60.0, 70.0, np.nan, 80.0, 90.0])
result = scores.describe()
print(result)
print("Rows:", scores.size, "Known:", scores.count())

Show a hint

Pass a list of probabilities [0.25, 0.5, 0.75].

Show one worked solution

scores = pd.Series([60.0, 70.0, np.nan, 80.0, 90.0])
result = scores.quantile([0.25, 0.5, 0.75])
print(result)

With the default linear interpolation, the quartiles are 67.5, 75.0, and 82.5.

Compare your explanations

Read. No. Its count is 4, while the Series has 5 entries.
Predict. 75.0, halfway between the two middle known values, 70 and 80.
Explain. Readers need to know how many observations support the average and how much information is missing.

One quick check

Which quantity includes a missing entry?

What you should be able to explain. Report completeness alongside summary statistics.

Your own assessment, separate from code checks

M15C04 · Explore · predict · practice

Correlation describes association

corr() computes pairwise correlations between numeric variables; its default method is Pearson correlation. Positive values indicate that higher values tend to occur together in a linear pattern. Correlation does not establish that one variable causes the other.

ReadWhy are only three columns selected before corr()?

PredictWhat values appear on the diagonal?

ExplainDoes a positive Study_Hours-Score correlation prove that studying longer caused these scores?

Read the example

df = students()
result = df[["Study_Hours", "Score", "Attendance"]].corr(method="pearson")
print(result.round(3))

Verified example output · pandas 2.2.3

             Study_Hours  Score  Attendance
Study_Hours        1.000  0.985       0.959
Score              0.985  1.000       0.983
Attendance         0.959  0.983       1.000

Watch for this. Missing entries are handled pairwise, so different correlations can use different numbers of observations.

Your turn

Calculate Spearman rank correlation for Study_Hours and Score. Store the two-by-two matrix in result.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

df = students()
result = df[["Study_Hours", "Score", "Attendance"]].corr(method="pearson")
print(result.round(3))

Show a hint

Change method to "spearman" after selecting the two columns.

Show one worked solution

df = students()
result = df[["Study_Hours", "Score"]].corr(method="spearman")
print(result.round(3))

Spearman compares ranks rather than raw values. The association is positive in this practice table; it still does not support a causal claim.

Compare your explanations

Read. They are numeric measurements relevant to this question; IDs and labels do not belong in this calculation.
Predict. 1.0 for each of these nonconstant columns correlated with itself.
Explain. No. Other factors and the way the data were collected could explain the association. These records are also fictional teaching data.

One quick check

What claim does correlation alone justify?

What you should be able to explain. Interpret a correlation as an association with limitations.

Your own assessment, separate from code checks

Previous: Map, apply, and vectorize Next: Group, summarize, and compare

Further detail: official documentation for this topic.