Module 21 · Extension

Work with time series

Distinguish calendar dates, observed records, windows, and changes over time.

Before you start: Parse dates and understand missing values and sorting.

Python loads from this site on your first run.

Each run starts with fresh data. Download any exported files before another run. Your written answers and code drafts are saved on this device; export your work to keep a backup.

M21C01 · Explore · predict · practice

Expose a missing day

A DatetimeIndex lets pandas work with calendar dates. asfreq("D") requests a daily index between the first and last observations. The missing October 4 row appears as unknown, not zero.

ReadWhat does freq="D" represent?

PredictHow many rows will result contain?

ExplainWhy not fill the missing day with zero automatically?

Read the example

df = attendance()
df["Date"] = pd.to_datetime(df["Date"], format="%Y-%m-%d")
result = df.set_index("Date")["Visitors"].sort_index().asfreq("D")
print(result)

Verified example output · pandas 2.2.3

Date
2026-10-01    20.0
2026-10-02    24.0
2026-10-03    18.0
2026-10-04     NaN
2026-10-05    30.0
2026-10-06    28.0
2026-10-07    32.0
Freq: D, Name: Visitors, dtype: float64

Watch for this. asfreq does not aggregate. Ensure the date index is unique before requesting a frequency.

Your turn

Build the daily Series and select only entries whose visitor count is missing. Store that filtered Series, retaining its date index, in result and print it.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

df = attendance()
df["Date"] = pd.to_datetime(df["Date"], format="%Y-%m-%d")
result = df.set_index("Date")["Visitors"].sort_index().asfreq("D")
print(result)

Show a hint

Filter the daily Series with its isna() mask.

Show one worked solution

df = attendance()
df["Date"] = pd.to_datetime(df["Date"], format="%Y-%m-%d")
daily = df.set_index("Date")["Visitors"].sort_index().asfreq("D")
result = daily[daily.isna()]
print(result)

Only 2026-10-04 is missing between the observed start and end dates.

Compare your explanations

Read. One index label per calendar day.
Predict. 7 calendar days, although the source has only 6 observations.
Explain. No record does not prove that nobody visited; the value is unknown.

One quick check

What can asfreq("D") reveal?

What you should be able to explain. A gap in records is not automatically a zero.

Your own assessment, separate from code checks

M21C02 · Explore · predict · practice

Resample into calendar bins

resample groups observations into time bins. Here each bin starts at a three-day boundary measured from the first day. Report the number of observations beside the total so a partial bin is visible.

ReadWhat is grouped by resample("3D")?

PredictWhat total is recorded for October 1 through October 3?

ExplainIs the total of 58 for the next bin a complete three-day total?

Read the example

df = attendance()
df["Date"] = pd.to_datetime(df["Date"], format="%Y-%m-%d")
s = df.set_index("Date")["Visitors"].sort_index()
groups = s.resample("3D")
result = pd.DataFrame({"Recorded_Total": groups.sum(min_count=1), "Observed_Days": groups.count()})
print(result)

Verified example output · pandas 2.2.3

            Recorded_Total  Observed_Days
Date                                     
2026-10-01              62              3
2026-10-04              58              2
2026-10-07              32              1

Watch for this. Time-bin labels and boundaries depend on frequency and options; state the reporting period.

Your turn

Resample to individual days and use sum(min_count=1) so October 4 stays missing. Store only the daily total Series, indexed by date, in result.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

df = attendance()
df["Date"] = pd.to_datetime(df["Date"], format="%Y-%m-%d")
s = df.set_index("Date")["Visitors"].sort_index()
groups = s.resample("3D")
result = pd.DataFrame({"Recorded_Total": groups.sum(min_count=1), "Observed_Days": groups.count()})
print(result)

Show a hint

Change "3D" to "D" and require at least one known value for a sum.

Show one worked solution

df = attendance()
df["Date"] = pd.to_datetime(df["Date"], format="%Y-%m-%d")
s = df.set_index("Date")["Visitors"].sort_index()
result = s.resample("D").sum(min_count=1)
print(result)

min_count=1 keeps a day with no observations unknown instead of reporting an empty sum as zero.

Compare your explanations

Read. Observations falling in successive three-day calendar bins.
Predict. 62: 20 + 24 + 18.
Explain. No. October 4 is missing, so it includes only the recorded counts for October 5 and 6.

One quick check

Why show Observed_Days beside totals?

What you should be able to explain. Time summaries need a coverage check.

Your own assessment, separate from code checks

M21C03 · Explore · predict · practice

Calculate a rolling mean

A rolling window calculates a statistic repeatedly as it moves through the data. On this daily grid, rolling(3) means three calendar rows. min_periods=3 requires three known counts, so windows crossing the missing day stay unknown.

ReadWhat does min_periods=3 require?

PredictWhat is the rolling mean on October 7?

ExplainWould rolling(3) mean three days on the original irregular index?

Read the example

df = attendance()
df["Date"] = pd.to_datetime(df["Date"], format="%Y-%m-%d")
daily = df.set_index("Date")["Visitors"].sort_index().asfreq("D")
result = pd.DataFrame({"Visitors": daily, "Mean_3_Days": daily.rolling(3, min_periods=3).mean()})
print(result.round(2))

Verified example output · pandas 2.2.3

            Visitors  Mean_3_Days
Date                             
2026-10-01      20.0          NaN
2026-10-02      24.0          NaN
2026-10-03      18.0        20.67
2026-10-04       NaN          NaN
2026-10-05      30.0          NaN
2026-10-06      28.0          NaN
2026-10-07      32.0        30.00

Watch for this. Sort time data before rolling. Otherwise the window follows an unintended order.

Your turn

Calculate a two-day rolling mean requiring two known daily counts. Store only the resulting mean Series, indexed by date, in result.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

df = attendance()
df["Date"] = pd.to_datetime(df["Date"], format="%Y-%m-%d")
daily = df.set_index("Date")["Visitors"].sort_index().asfreq("D")
result = pd.DataFrame({"Visitors": daily, "Mean_3_Days": daily.rolling(3, min_periods=3).mean()})
print(result.round(2))

Show a hint

Apply rolling(2, min_periods=2).mean() to the daily Series and save that Series directly.

Show one worked solution

df = attendance()
df["Date"] = pd.to_datetime(df["Date"], format="%Y-%m-%d")
daily = df.set_index("Date")["Visitors"].sort_index().asfreq("D")
result = daily.rolling(2, min_periods=2).mean()
print(result.round(2))

The last two-day mean is also 30, using only October 6 and 7. Earlier windows differ.

Compare your explanations

Read. At least three non-missing observations within the current window.
Predict. 30: (30 + 28 + 32) / 3.
Explain. No. It would mean three observed rows, which can span more than three days.

One quick check

Without a regular daily grid, rolling(3) means what?

What you should be able to explain. Window size and missing-data rules both affect a moving average.

Your own assessment, separate from code checks

M21C04 · Explore · predict · practice

Compare with a previous period

shift(1) places the previous row value beside the current row. pct_change(fill_method=None) calculates fractional change without filling gaps. Multiplying by 100 expresses that change as a percentage.

ReadWhy is the first change missing?

PredictWhat percentage change is shown on October 2?

ExplainWhy is the October 5 change unknown?

Read the example

df = attendance()
df["Date"] = pd.to_datetime(df["Date"], format="%Y-%m-%d")
daily = df.set_index("Date")["Visitors"].sort_index().asfreq("D")
result = pd.DataFrame({"Visitors": daily, "Previous_Day": daily.shift(1), "Change_Percent": daily.pct_change(fill_method=None) * 100})
print(result.round(2))

Verified example output · pandas 2.2.3

            Visitors  Previous_Day  Change_Percent
Date                                              
2026-10-01      20.0           NaN             NaN
2026-10-02      24.0          20.0           20.00
2026-10-03      18.0          24.0          -25.00
2026-10-04       NaN          18.0             NaN
2026-10-05      30.0           NaN             NaN
2026-10-06      28.0          30.0           -6.67
2026-10-07      32.0          28.0           14.29

Watch for this. Percent change from a previous zero is undefined; inspect zero denominators.

Your turn

Calculate the signed difference in visitor counts from two calendar days earlier using shift(2). Store the difference Series, indexed by date, in result.

For the code check, keep these result variable names: result. Different valid approaches are welcome.

Your Python code

df = attendance()
df["Date"] = pd.to_datetime(df["Date"], format="%Y-%m-%d")
daily = df.set_index("Date")["Visitors"].sort_index().asfreq("D")
result = pd.DataFrame({"Visitors": daily, "Previous_Day": daily.shift(1), "Change_Percent": daily.pct_change(fill_method=None) * 100})
print(result.round(2))

Show a hint

Subtract the shifted daily Series from the original daily Series.

Show one worked solution

df = attendance()
df["Date"] = pd.to_datetime(df["Date"], format="%Y-%m-%d")
daily = df.set_index("Date")["Visitors"].sort_index().asfreq("D")
result = daily - daily.shift(2)
print(result)

October 7 is two visitors higher than October 5. Comparisons involving missing dates stay unknown.

Compare your explanations

Read. There is no previous value in this series for comparison.
Predict. 20%, from 20 to 24.
Explain. The previous calendar day, October 4, has no known visitor count.

One quick check

Why multiply pct_change by 100?

What you should be able to explain. Be explicit about the time interval and denominator.

Your own assessment, separate from code checks

Previous: Read and write files Next: Explore richer structures

Further detail: official documentation for this topic.