Subsetting and filtering data

df[rows, columns] slices a data frame both ways at once: conditions like df$screen > 180 pick rows, name vectors pick columns, and subset() spells the same thing in readable English.

9 min read · 9 cards · 2 checks

Read in: English · हिन्दी · ગુજરાતી


Theory

The dean wants a shortlist

The dean reads the CampusPulse summary and narrows in: "show me just the hostellers above 180 minutes of screen time: roll numbers and their minutes, nothing else."

A row condition AND a column selection, in one request. You have answered this shape twice: SQL's WHERE + SELECT, pandas' masks + double brackets.

R's spelling is one pair of square brackets with a comma in the middle, and that comma carries the whole grammar.

Theory

The two-part order slip

Think of df[ ___ , ___ ] as an order slip with two blanks:

  • LEFT of the comma: which rows (which students).
  • RIGHT of the comma: which columns (which facts about them).
  • Leave a blank empty and it means ALL of that dimension.

df[rows, ]: chosen students, every fact. df[, cols]: every student, chosen facts. The comma is the slip's dividing line: forget it and the kitchen misreads the order.

Practical

The bracket grammar, then subset()

survey <- read.csv("survey.csv")   # roll, city, stay, screen

# rows by NUMBER, all columns (empty after comma)
survey[1:5, ]

# rows by CONDITION: the workhorse
survey[survey$screen > 180, ]

# columns by NAME, all rows (empty before comma)
survey[, c("roll", "screen")]

# both at once: the dean's request
survey[survey$stay == "hostel" & survey$screen > 180,
       c("roll", "screen")]

# membership test with %in%
survey[survey$city %in% c("Surat", "Navsari"), ]

# subset(): same result, reads like English (no survey$ prefixes)
subset(survey, stay == "hostel" & screen > 180,
       select = c(roll, screen))

# keep a result by assigning it
heavy <- subset(survey, screen > 180)
nrow(heavy)

Theory

Conditions: the same logic, R-spelled

Row conditions are logical vectors (the syntax lesson's survey$screen > 180) used as row-pickers:

  • == tests equality (single = assigns: the classic slip).
  • & and, | or, ! not: element-wise, each side in its own complete comparison: survey$screen > 180 & survey$stay == "hostel".
  • %in% tests membership in a set: R's IN.

Translation table: rows-condition = SQL WHERE = pandas mask; the columns blank / select = SQL's SELECT list. Third language, same two moves.

Quiz

A student types survey[survey$screen > 180] (no comma) expecting the heavy screen-time rows. What did R understand?

  1. Without the comma, R treats it as COLUMN selection, not row filtering: the two-blank slip needs its comma: survey[survey$screen > 180, ]
  2. It works identically: the comma is optional style
  3. It deletes the matching rows
  4. R sorts the data frame by screen time
Show the answer

Without the comma, R treats it as COLUMN selection, not row filtering: the two-blank slip needs its comma: survey[survey$screen > 180, ]

Single-argument brackets on a data frame select columns, so R tries to use the logical vector as a column-picker: wrong columns or an error, never the intended rows. The row/column grammar lives entirely in that comma: left = rows, right = columns, empty = all. Writing df[condition, ] with the trailing comma-space is the habit that makes this bug impossible.

Think first

Translate the dean, three ways

Write the dean's request (hostellers with screen > 180; only roll and screen) in: (1) R brackets, (2) R subset(), (3) the SQL you learned in BCA303. Sketch all three before tapping.

Show the answer

1. survey[survey$stay == "hostel" & survey$screen > 180, c("roll", "screen")]

2. subset(survey, stay == "hostel" & screen > 180, select = c(roll, screen))

3. SELECT roll, screen FROM survey WHERE stay = 'hostel' AND screen > 180;

Three spellings, one thought: filter rows, choose columns. If the three feel like the same sentence in different accents, the concept has landed: that recognition is what interviews test.

Watch out

The four filtering slips

The missing comma: df[condition, ]: left blank rows, right blank columns, always.

= vs ==: conditions compare with ==; a single = inside brackets is an error or an accident.

'and' does not exist: R wants & and |, one ampersand (&& is for single values, not vectors).

Unassigned results vanish: filtering prints and forgets: keep it with heavy <- subset(...).

Theory

Filtering feeds everything ahead

Every statistic you compute from now on is really "statistic OF a subset": mean screen time OF hostellers, SD OF year 3, boxplot OF each city. Slicing is the preposition of statistics. Next lessons: adding and renaming columns (the survey gains derived variables), then cleaning: where the NA values your conditions quietly skipped finally face justice.

Summary

Key takeaways

  • df[rows, cols]: left of the comma picks rows, right picks columns, EMPTY means all.
  • Rows by number (1:5), by condition (df$screen > 180), columns by name vector.
  • Combine conditions with & | !, compare with ==, membership with %in%.
  • subset(df, condition, select = cols) says the same thing without df$ prefixes.
  • The pattern = SQL WHERE + SELECT = pandas mask + column list: third language, same idea.
  • Assign filtered results or they print and vanish.
  • Memory hook: the two-blank order slip, divided by its comma.

Study this properly

This page is the lesson to read. In Gri-Learn the same topic is a graded deck: the self-checks are scored and your weak topics are tracked. Free to start.

Start this topic

Already have an account? Sign in

More from Data Filtering and cleaning

Gri-Learn · syllabus-mapped B.C.A. lessons in English, Hindi and Gujarati

Subsetting and filtering data · Statistical Methods and Data Analysis (MDC-03) · Gri-Learn