Boolean Masks & Filtering¶

Boolean indexing is the most powerful feature for data wrangling — pick out, count, modify, or summarize elements that match conditions.

A boolean mask¶

A boolean array of the same shape as your data:

import numpy as np

scores = np.array([85, 42, 77, 95, 33, 60, 88, 70, 50])

mask = scores >= 60
print("mask :", mask)
print("pass :", scores[mask])
print("count:", mask.sum())

Filter with a mask — `arr[mask]`¶

Returns only the elements where mask is True:

import numpy as np

a = np.array([1, 5, 3, 9, 2, 8, 4, 7])

print(a[a > 4])           # [5 9 8 7]
print(a[a % 2 == 0])      # [2 8 4]
print(a[a != 5])          # everything except 5

Combining conditions — `&`, `|`, `~`¶

Use bitwise operators (NOT Python's and/or):

import numpy as np

a = np.array([1, 5, 3, 9, 2, 8, 4, 7])

# AND — between 3 and 7
print(a[(a >= 3) & (a <= 7)])      # [5 3 4 7]

# OR — very small or very large
print(a[(a < 2) | (a > 7)])         # [1 9 8]

# NOT
print(a[~(a > 5)])                  # [1 3 2 4]

Always wrap each condition in parens — & / | have lower precedence than > / <.

Counting — boolean sums¶

A True is 1, False is 0 — so .sum() counts:

import numpy as np

scores = np.array([85, 42, 77, 95, 33, 60, 88, 70, 50])

print("Passed:", (scores >= 60).sum())
print("A grades (>=90):", (scores >= 90).sum())
print("Between 50-70:", ((scores >= 50) & (scores < 70)).sum())

# As percentages
print(f"Pass rate: {(scores >= 60).mean() * 100:.1f}%")

(condition).mean() gives the fraction of True values — handy.

Modify via mask — conditional assignment¶

import numpy as np

scores = np.array([85, 42, 77, 95, 33, 60])
scores_clean = scores.copy()

# Anyone below 50 gets pulled up to 50 (curve)
scores_clean[scores_clean < 50] = 50
print(scores_clean)

# Cap anything > 90 at 90
scores_clean[scores_clean > 90] = 90
print(scores_clean)

`np.where` — conditional pick¶

np.where(cond, x, y) returns x where cond is True, else y:

import numpy as np

a = np.array([3, -1, 4, -5, 2, -8])

# Absolute value (manual)
print(np.where(a < 0, -a, a))         # [3 1 4 5 2 8]

# Replace negatives with 0
print(np.where(a < 0, 0, a))

# Pass/fail labels
scores = np.array([85, 42, 77, 95, 33, 60])
labels = np.where(scores >= 60, "pass", "fail")
print(labels)

Multi-condition with `np.select`¶

For if / elif / else-style branching across an array:

import numpy as np

scores = np.array([95, 85, 75, 65, 55, 45])

conditions = [
    scores >= 90,
    scores >= 80,
    scores >= 70,
    scores >= 60,
]
choices = ["A", "B", "C", "D"]

grades = np.select(conditions, choices, default="F")
print(grades)

Filtering 2D arrays¶

The mask just needs the same shape:

import numpy as np

m = np.array([
    [1, 2, 3],
    [4, 5, 6],
    [7, 8, 9],
])

# Returns a FLAT 1D array of the matching elements
print(m[m > 4])

# Selecting rows where a column condition is True
print(m[m[:, 0] > 2])         # rows where first col > 2

`np.isnan` — find missing values¶

import numpy as np

data = np.array([1.0, np.nan, 3.0, np.nan, 5.0])

print("missing mask:", np.isnan(data))
print("count missing:", np.isnan(data).sum())

# Replace NaN with 0
filled = np.where(np.isnan(data), 0, data)
print("filled:", filled)

# Or use np.nan_to_num — same idea, more direct
print("nan_to_num:", np.nan_to_num(data, nan=0.0))

nan math is special — np.nan + 1 is nan, np.nan == np.nan is False. Always test with np.isnan.

`np.isfinite`, `np.isinf`¶

import numpy as np

x = np.array([1.0, np.inf, -np.inf, np.nan, 5.0])

print("finite     :", np.isfinite(x))
print("infinite   :", np.isinf(x))
print("nan        :", np.isnan(x))

# Remove non-finite values
clean = x[np.isfinite(x)]
print("clean:", clean)

`np.isin` — set membership¶

import numpy as np

a = np.array([1, 2, 3, 4, 5, 6, 7, 8])
allowed = [2, 4, 6, 8]

mask = np.isin(a, allowed)
print(a[mask])         # the allowed ones
print(a[~mask])        # the rest

`np.nonzero` / `np.where(cond)` — indices of True¶

import numpy as np

a = np.array([0, 3, 0, 5, 0, 0, 7])

print("indices:", np.nonzero(a)[0])      # [1 3 6]
print("equivalent:", np.where(a != 0)[0])

A realistic example — clean and analyze a dataset¶

import numpy as np

rng = np.random.default_rng(0)

# Fake "patient data"
ages    = rng.integers(0, 100, size=100)
heights = rng.normal(170, 10, size=100)
weights = rng.normal(70, 15, size=100)

# Inject some bad data
ages[5] = -1            # impossible
heights[10] = np.nan    # missing
weights[20] = 500       # outlier

# Find and report bad rows
bad_age    = ages < 0
bad_height = np.isnan(heights)
bad_weight = weights > 200

bad_anywhere = bad_age | bad_height | bad_weight
print(f"Bad rows: {bad_anywhere.sum()} / {len(ages)}")
print(f"Indices : {np.where(bad_anywhere)[0]}")

# Keep only valid rows
valid = ~bad_anywhere
print(f"\nMean weight of valid rows: {weights[valid].mean():.1f}")
print(f"Mean height of valid rows: {heights[valid].mean():.1f}")

This is the essence of data cleaning — a single mask captures every quality issue.

Cheatsheet¶

Want	Use
Pick elements matching a condition	`arr[arr > 5]`
Combine conditions (AND)	`arr[(arr > 5) & (arr < 10)]`
OR	`arr[(arr < 0) \| (arr > 100)]`
NOT	`arr[~(arr > 5)]`
Count matches	`(arr > 5).sum()`
Fraction matching	`(arr > 5).mean()`
Any match	`(arr > 5).any()`
All match	`(arr > 5).all()`
Conditional pick	`np.where(cond, x, y)`
Multi-condition	`np.select([c1, c2], [v1, v2], default=...)`
Find missing	`np.isnan(arr)`
Set membership	`np.isin(arr, allowed)`
Indices where True	`np.where(cond)[0]`

Common pitfalls¶

❗ and / or instead of & / | — Python's and/or doesn't work element-wise. Always use & / |.
❗ Forgetting parens — arr > 3 & arr < 8 parses as arr > (3 & arr) < 8. Use (arr > 3) & (arr < 8).
❗ == with NaN — np.nan == np.nan is False. Use np.isnan(arr).
❗ Modifying via mask doesn't work for fancy assignment — arr[arr > 5] += 1 works but may behave oddly with duplicates.
❗ np.where(cond) returns a tuple — even for 1D arrays. Use np.where(cond)[0] for plain indices.

Practice¶

What does this print?

Expected: 3

import numpy as np
a = np.array([1, 5, 3, 9, 2, 8, 4])
print((a > 4).sum())

Filter values strictly between 3 and 8 (exclusive on both ends)

Expected: [5 4 7]

import numpy as np
a = np.array([1, 5, 3, 9, 2, 8, 4, 7])
print(a[(a > 3) | (a < 8)])     # bug: | (OR) catches everything — use &

Quiz — Quick check¶

What you remember

Q1. Why does arr[a > 3 & a < 8] give a confusing error?

Without parens, & binds tighter than < / > — Python parses it as a > (3 & a) < 8
& doesn't work in NumPy
a > 3 returns a scalar
Mask must be 2D

Why: Operator precedence. The bitwise & happens before comparisons. Always wrap each condition in parens: (a > 3) & (a < 8).

Q2. What does (arr > 0).mean() give?

The mean of positive values
The fraction of elements that are positive (between 0 and 1)
The number of positive values
A boolean array

Why: Booleans are 1/0 — the mean of a boolean array is the proportion of True values. Same pattern: (arr > 0).sum() gives a count.

Q3. Why does np.nan == np.nan return False?

Bug in NumPy
By IEEE 754 standard — NaN is "not equal to anything, including itself"
Only when using float32
It returns nan, not False

Why: This is the IEEE float standard, not a NumPy quirk. To check for NaN, use np.isnan(arr). Same in Pandas, R, JavaScript, etc.

Common doubts¶

Why must I use & instead of and on arrays?

Python's and returns the first falsy operand (or the second if both truthy) — it expects single booleans. With a boolean array, Python doesn't know what "the whole array" should resolve to, so it raises ValueError: ambiguous truth value. & is element-wise — produces a new boolean array.

How is np.where(cond, x, y) different from boolean masking arr[cond]?

Boolean masking selects elements and returns a smaller array (arr[arr > 5]). np.where(cond, x, y) constructs a same-shape array picking from x where True and y where False. Use masking to filter; use where to transform.

What happens with np.isnan on a non-float array?

Raises TypeError. NaN is a float concept — integer arrays can't store NaN. If you need missing-value semantics for ints, use np.ma.masked_array, Pandas (which uses pd.NA), or convert to float first.

What's next¶

→ Real-World Examples