Pandas Indexing: loc vs iloc, Boolean Masks, Copy-on-Write & MultiIndex

Q: What's the difference between loc and iloc?

.loc selects by labels (index values and column names), or boolean arrays, and label slices include the end label. .iloc selects by integer positions, like Python lists, and position slices exclude the end. Use .loc for meaningful keys and conditions, and .iloc when you truly mean "the first N rows".

Q: How do I fix SettingWithCopyWarning?

Don't chain indexing when assigning. Use a single .loc[row_selector, column] = value on the DataFrame you want to modify, or create an explicit copy with .copy() if you intend to work on a separate subset. With Copy-on-Write (default in pandas 3), the warning disappears, but chained assignment still doesn't modify the original.

Q: How do I filter rows by multiple conditions?

Combine boolean Series with & (and), | (or), and ~ (not), wrapping each condition in parentheses: df.loc[(df.a 1) & (df.b == "x")]. Or use df.query("a 1 and b == 'x'") for readability.

Pandas Indexing & Selection

Almost every pandas workflow starts with selecting data: specific columns, rows matching a condition, a date range, a single cell to fix. Pandas offers several ways to do it ([], .loc, .iloc, boolean masks, .query()), and choosing the wrong one produces some of the most confusing behavior in the library: selecting by label when you meant position, silent no-op assignments, and the infamous SettingWithCopyWarning.

Modern pandas (2.x with Copy-on-Write, the default in 3.0) makes the rules simpler and more predictable. This page covers how DataFrames are structured, how to select precisely, and how to modify data safely.

TL;DR

A DataFrame is a table of columns (each a Series) sharing a row index.
.loc[rows, cols] selects by label (inclusive slices); .iloc[rows, cols] by integer position (exclusive slices).
Boolean masks filter rows: df.loc[(df.status == "paid") & (df.total > 100)]. Use &, |, and ~ with parentheses.
.query("status == 'paid' and total > 100") offers readable filtering; .assign() adds columns in chains.
Set values with .loc[mask, "col"] = value in one step. Never chain indexing for assignment.
Copy-on-Write makes every selection behave like a copy. Understand it to avoid silent no-op updates.

Quick Example

Core Concepts

DataFrame Anatomy

Columns: each a Series with one dtype (int64, float64, bool, datetime64[ns], category, string, object, or Arrow-backed types).
Index: row labels, a RangeIndex (0..n-1) by default, or meaningful keys (IDs, dates) via set_index.
Alignment: operations between Series and DataFrames align on index labels, not positions. It's powerful, and surprising when indexes differ.

The Selection Methods

Rule of thumb: use .loc for labels and conditions, .iloc for positions, and be explicit.

Boolean Masks

A comparison on a Series produces a boolean Series; indexing with it keeps True rows:

Use &, |, and ~ (not and/or/not), and wrap each condition in parentheses because of operator precedence. Useful helpers: .isin(), .between(), .isna()/.notna(), and the .str and .dt accessors.

Setting Values and Copy-on-Write

With Copy-on-Write (CoW) (optional in 2.x, the default in pandas 3.0), any DataFrame or Series derived from another behaves as an independent copy. Modifying it never changes the original, and memory is shared lazily until a write occurs. Consequences:

Chained assignment never works: df[df.x > 0]["y"] = 1 modifies a temporary object and is silently lost (older pandas warned with SettingWithCopyWarning).
Assign through one .loc call on the object you want to change: df.loc[df.x > 0, "y"] = 1.
Methods return new objects; inplace=True offers no real benefit and is discouraged.

Index and MultiIndex

set_index("col") / reset_index() move columns to and from the index.
A meaningful index enables fast label lookups, automatic alignment, and time-series slicing with a DatetimeIndex. See time series.
A MultiIndex (hierarchical index) arises from grouping and pivoting. Select with tuples (df.loc[("DE", "2026-09")]), xs(), or pd.IndexSlice. Flatten with reset_index() when it gets awkward.
Sort the index (sort_index()) for efficient slicing and to avoid performance warnings.

Missing Data

Pandas represents missing values as NaN, None, NaT (datetimes), or pd.NA (nullable dtypes). Tools: isna(), notna(), dropna(subset=…), fillna(value or method), and interpolate(). Nullable dtypes (Int64, boolean, string, or Arrow-backed ones) keep integers as integers when values are missing, instead of silently converting them to float.

Best Practices

Be Explicit With loc and iloc

Plain df[...] changes meaning depending on the argument type. .loc and .iloc make intent clear, and avoid label-vs-position bugs, especially with integer indexes.

Use Method Chaining

df.query(...).assign(...).loc[:, cols].sort_values(...) builds readable transformation pipelines without temporary variables or in-place mutation, and it works well with Copy-on-Write.

Set Correct dtypes Early

Parse dates on load, convert categorical strings to category, and use nullable or Arrow dtypes. Correct types make selection faster and prevent comparisons of strings against numbers. See pandas performance.

Check Shapes and Samples

After filtering or joining, check shape, head(), and value_counts() to confirm you selected what you intended. Silent over- or under-filtering is a common analysis bug.

Common Mistakes

Chained Assignment

Using and/or With Series

df[(df.a > 1) and (df.b < 5)] raises "The truth value of a Series is ambiguous". Use & and |, with parentheses.

Confusing Inclusive and Exclusive Slices

df.loc["a":"c"] includes "c"; df.iloc[0:3] excludes position 3. Mixing them up drops or adds a row at the boundary.

FAQ

What's the difference between loc and iloc?

.loc selects by labels (index values and column names), or boolean arrays, and label slices include the end label. .iloc selects by integer positions, like Python lists, and position slices exclude the end. Use .loc for meaningful keys and conditions, and .iloc when you truly mean "the first N rows".

How do I fix SettingWithCopyWarning?

Don't chain indexing when assigning. Use a single .loc[row_selector, column] = value on the DataFrame you want to modify, or create an explicit copy with .copy() if you intend to work on a separate subset. With Copy-on-Write (default in pandas 3), the warning disappears, but chained assignment still doesn't modify the original.

How do I filter rows by multiple conditions?

Combine boolean Series with & (and), | (or), and ~ (not), wrapping each condition in parentheses: df.loc[(df.a > 1) & (df.b == "x")]. Or use df.query("a > 1 and b == 'x'") for readability.

When should I set an index?

When you frequently look up rows by a key, work with time series, or want automatic alignment between datasets. Keep a plain RangeIndex when rows have no natural key, or when you mostly filter by column conditions.