Audio Masking and Psychoacoustic Models: How Compression Keeps What We Don’t Hear

Modern audio compression does something that feels almost magical: it removes large amounts of data while keeping the sound nearly the same to our ears. This works because human hearing is not perfect. Some sounds hide others, and our brain simply never notices what gets buried.

This phenomenon is called masking, and it is the foundation of psychoacoustic audio compression used in formats like MP3 and AAC.

What Is Audio Masking?

Masking happens when one sound makes another sound harder or impossible to hear. The stronger sound is the masker, and the weaker sound becomes masked.

There are two main types of masking:

• Frequency (spectral) masking
• Time (temporal) masking

Both limit what we can actually perceive, even if the signal physically exists.

Frequency Masking

Frequency masking occurs when a loud tone hides quieter tones at nearby frequencies.

For example:

• A loud sound at 1 kHz
• Quieter sounds slightly above or below 1 kHz

The louder sound raises the hearing threshold around it, making nearby frequencies inaudible unless they are strong enough.

Important details:

• Masking is stronger toward higher frequencies than lower ones
• The masking effect spreads across a frequency region, not just a single point

This is why compressors do not treat every frequency independently — they work in bands.

Time Masking

Time masking happens when sounds close together in time interfere with each other.

There are two forms:

Forward Masking

A loud sound makes it harder to hear quiet sounds that follow shortly after it.

• Lasts for a few milliseconds
• Happens because the ear and brain need time to recover

Backward Masking

A loud sound can mask a quiet sound that occurred just before it.

• Much shorter effect
• Still strong enough to influence perception

Together, these effects mean the ear has limited time resolution, especially around loud events like drum hits or consonants in speech.

Why Masking Enables Compression

Digital audio normally stores everything:

• Loud parts
• Quiet parts
• Sounds that are technically present but inaudible

Compression removes the parts that fall below the masking threshold, meaning the listener could not hear them anyway.

This allows:

• Fewer bits to be stored
• Smaller file sizes
• Similar perceived sound quality

Instead of compressing blindly, codecs use psychoacoustic models to decide what can be safely discarded.

Psychoacoustic Models: Predicting What We Hear

A psychoacoustic model is a mathematical approximation of human hearing. It estimates:

• Which frequencies are masked
• When masking occurs in time
• How much noise can be added without being noticed

The codec analyzes short blocks of audio and performs a frequency transform to obtain the spectrum.

Once the spectrum is known, the model estimates masking behavior across frequencies and time.

Bark Scale and Critical Bands

Human hearing does not perceive frequency linearly. Instead, it groups frequencies into critical bands, often modeled using the Bark scale.

Key ideas:

• Each Bark band represents a frequency region processed together by the ear
• Masking mostly happens within and near the same Bark band
• Fine frequency details inside a band matter less perceptually

So instead of analyzing thousands of tiny frequency bins independently, codecs group them into perceptually meaningful bands.

Calculating the Masking Threshold

For each Bark band, the psychoacoustic model estimates:

• The energy of strong tones
• How far their masking spreads
• How much noise is allowed without being audible

This creates a masking threshold curve across the frequency spectrum.

Anything below this curve is assumed to be inaudible and can be:

• Removed
• Quantized very coarsely
• Replaced by low-level noise

The goal is not perfect accuracy, but staying below what the ear can detect.

Quantization Based on Masking

Once the masking threshold is known, the codec decides how many bits to allocate to each band.

Process in simple terms:

1. Analyze spectrum
2. Estimate masking threshold per band
3. Allocate more bits where sound is clearly audible
4. Allocate fewer bits where masking hides errors

This is called perceptual bit allocation.

If quantization noise stays below the masking threshold, it becomes perceptually invisible, even though it technically alters the signal.

Repeating the Process Over Time

Audio is dynamic, so this analysis repeats:

• For every short time window
• Across all frequency bands

Each frame may have different masking conditions:

• Loud drums mask more noise
• Quiet passages require more precision

The codec constantly adapts its bit usage to match what the ear is likely to notice at each moment.

Why It Still Sounds Good

Even though compression removes information, it preserves:

• Strong tones
• Clear speech formants
• Important transients

What gets removed is mostly:

• Very quiet background details
• Noise that is masked by louder sounds
• Frequencies that blend into nearby tones

The brain reconstructs a convincing sound image because the remaining signal still matches the ear’s expectations.

The Balance Between Math and Perception

Audio compression is not about perfect signal reproduction. It is about perceptual transparency — sounding the same to a human listener, not being numerically identical.

By combining:

• Frequency analysis
• Bark-scale grou