Work efficiently and reproducibly

memory_usage(deep=True) estimates memory including text stored by object columns. Repeated labels may use less memory as categories. The benefit depends on the data, so measure it instead of assuming.

labels = pd.Series(["RAK", "Dubai"] * 500, dtype=object)
category = labels.astype("category")
result = pd.Series({"Object_bytes": labels.memory_usage(deep=True), "Category_bytes": category.memory_usage(deep=True)})
print(result)

Object_bytes      53132
Category_bytes     1346
dtype: int64

Watch for this. This is a small storage demonstration, not a benchmark of all pandas workloads.

labels = pd.Series([f"ID_{i}" for i in range(1000)], dtype=object)
category = labels.astype("category")
result = pd.Series({"Object_bytes": labels.memory_usage(deep=True), "Category_bytes": category.memory_usage(deep=True)})
print(result)

What you should be able to explain. Choose data types based on meaning and measurement.

usecols avoids reading columns that the question does not need. Explicit dtypes make the import contract clearer. This small file demonstrates a habit that becomes useful with larger files.

result = pd.read_csv(DATA_DIR / "students.csv", usecols=["Student_ID", "Program", "Score"], dtype={"Student_ID": "string", "Program": "category"})
print(result.head())
print(result.dtypes)

  Student_ID Program  Score
0       S001      AI     88
1       S002      IT     72
2       S003      AI     91
3       S004      IT     65
4       S005      IT     79
Student_ID    string[python]
Program             category
Score                  int64
dtype: object

Watch for this. A file preview can miss problems that occur later in the file.

result = pd.read_csv(DATA_DIR / "students.csv", usecols=["Student_ID", "Attendance"], nrows=4, dtype={"Student_ID": "string"})
print(result)

What you should be able to explain. Read the fields and rows needed for the current step.

chunksize reads a CSV in pieces. A total and a non-missing count can be accumulated across pieces without keeping all rows. Divide the overall total by the overall count to calculate the overall mean.

total = 0
count = 0
for chunk in pd.read_csv(DATA_DIR / "students.csv", usecols=["Score"], chunksize=5):
    total += chunk["Score"].sum()
    count += chunk["Score"].count()
result = total / count
print("Known scores:", count)
print("Mean score:", round(result, 2))

Known scores: 12
Mean score: 78.92

Watch for this. Not every operation is easy to compute in chunks. A global sort or median needs a different strategy.

result = 0
for chunk in pd.read_csv(DATA_DIR / "students.csv", usecols=["Score"], chunksize=4):
    result += chunk["Score"].ge(80).sum()
print("Students with Score >= 80:", result)

What you should be able to explain. Combine summaries using their correct mathematical weights.

sample chooses rows randomly. random_state makes the selection repeatable for this data and environment. Reproducibility does not make a tiny sample representative of the population.

df = students()
result = df.sample(n=3, random_state=7)
repeat = df.sample(n=3, random_state=7)
print(result[["Student_ID", "Campus", "Score"]])
print("Same sample:", result.equals(repeat))

   Student_ID Campus  Score
7        S008  Dubai     58
10       S011    RAK     68
2        S003    RAK     91
Same sample: True

Watch for this. Document input data, row order, library versions, and seed when an exact sample matters.

df = students()
result = df.groupby("Campus", group_keys=False).sample(n=2, random_state=7)
print(result[["Student_ID", "Campus", "Score"]])
print(result["Campus"].value_counts())

What you should be able to explain. Repeatability and representativeness are different goals.