Theory
Sixty lines become one
Last lesson's csv-module code read marks.csv with a with-block, a reader, a header skip, a loop, and int() conversions: honest work, a dozen lines before any analysis began.
Now watch the same job in the tool data analysts actually use:
df = pd.read_csv('marks.csv')
One line. Header understood, numbers already numbers, the whole table sitting in one object called a DataFrame. That object, and the pandas library behind it, powers the rest of this subject.
Theory
The whole sheet on your desk
The csv module was a postman handing you one envelope at a time: fine for sorting mail, painful for analysis.
pandas lifts the entire spreadsheet onto your desk as a single object: every row, every column, labelled and typed. Want an average? Ask the sheet. Want it saved as Excel? Tell the sheet. You stop thinking in lines-of-a-file and start thinking in tables, the same mental model SQL gave you.
Theory
pandas, numpy, DataFrame: who is who
numpy: the engine: lightning-fast arrays and maths (import numpy as np).
pandas: the table layer built on numpy (import pandas as pd). The aliases pd and np are universal convention: use them in exams too.
DataFrame: pandas' central object: a 2-D table with an index (row labels, 0,1,2,... by default), named columns, and a dtype per column (int64, float64, object for text).
Both libraries are third-party: pip install pandas numpy once per machine, unlike the built-in csv and sqlite3.
Practical
CSV in, Excel out (and back)
import pandas as pd
# load: one line, header + types handled
df = pd.read_csv('marks.csv')
print(df.shape) # (60, 3) rows, columns
print(df.columns) # Index(['roll', 'subject', 'score'], ...)
print(df.dtypes) # roll int64, subject object, score int64
print(df) # the table itself, index on the left
# write: CSV and Excel (Excel needs: pip install openpyxl)
df.to_csv('backup.csv', index=False) # index=False: no row-number column
df.to_excel('report.xlsx', index=False)
# Excel in, too:
sheet = pd.read_excel('attendance.xlsx')
# the numpy engine underneath, when you need the raw array:
arr = df.to_numpy()
print(arr[0]) # [101 'DBMS' 78]
Quiz
What does pd.read_csv('marks.csv') return, and how do its values differ from csv.reader's?
- A DataFrame: one table object with named, TYPE-INFERRED columns (score is int64, not the string '78')
- A list of lists of strings, same as csv.reader
- A dictionary per row, same as DictReader
- A generator that must be looped before any data exists
Show the answer
A DataFrame: one table object with named, TYPE-INFERRED columns (score is int64, not the string '78')
read_csv parses the whole file into a DataFrame and infers each column's type from its content: score arrives as numbers ready for maths, no int() conversions. That type inference is the practical leap over csv.reader's everything-is-a-string world. Options B and C describe the previous lesson's tools; a DataFrame is eagerly loaded, not a generator.
Think first
The mystery column
A student saves with df.to_csv('backup.csv') (no index=False), then reloads it tomorrow. The DataFrame now has a strange extra first column named 'Unnamed: 0' holding 0, 1, 2, ... Before tapping: where did it come from, and what is the fix?
Show the answer
to_csv wrote the DataFrame's index (the 0,1,2,... row labels) into the file as a real column; on reload, pandas found an unlabelled column and named it 'Unnamed: 0'. Save again tomorrow and you get two of them: the columns breed.
Fix: df.to_csv('backup.csv', index=False): the single most-typed pandas argument. Unless your index carries real meaning, always switch it off on export.
Watch out
Setup and judgement traps
pandas is not built in: ModuleNotFoundError means pip install pandas, and read_excel additionally wants openpyxl: worth one line in setup questions.
index=False on every export unless you know why not.
Do not reach for pandas for everything: appending one log line wants last lesson's open(); streaming a million rows row-by-row wants csv.reader. pandas earns its weight when you ANALYSE tables.
Theory
The bridge you have been building
Look at the pipeline you now own: SQL shapes data in college.db, Unit 2 exports it as CSV, and read_csv lifts it into a DataFrame (pd.read_sql exists too, connecting pandas straight to your sqlite3 connection: one line worth knowing). Next lessons: slicing the DataFrame (rows, columns, conditions) and computing the statistics ResultDesk was born for.
Summary
Key takeaways
- import pandas as pd, import numpy as np: universal aliases; both are pip-installed, not built in.
- A DataFrame = a 2-D table in memory: index, named columns, one inferred dtype per column.
- pd.read_csv / pd.read_excel load a whole file in one line; df.to_csv / df.to_excel write back.
- Always export with index=False, or the row numbers become a mystery 'Unnamed: 0' column.
- df.shape, df.columns, df.dtypes are the first three inspections on any loaded table.
- numpy is the fast array engine underneath; df.to_numpy() exposes it.
- Memory hook: the whole sheet on your desk, not one envelope at a time.