A CD stores every second of music as 1,411,200 bits. An MP3 at 128 kbit/s keeps about one bit in eleven, and on most music most listeners hear little difference. Three ideas make that work: turn sound into numbers, throw away what ears ignore, and store the music as a few big frequency numbers instead of many small samples.
A microphone turns air pressure into a voltage that rises and falls smoothly. A computer can only store a list of numbers, so a converter measures that voltage at evenly spaced instants. That's sampling, and the sample rate is how many measurements it takes per second, in hertz (Hz). Each measurement is then rounded to one of a fixed set of levels. That's quantization, and the bit depth is how many bits store each sample: b bits give 2b levels. Sample too slowly and a high tone masquerades as a lower one. Use too few bits and the rounding becomes audible hiss.
How to read this: The faint green tone is the real sound; the amber dots are the measurements the computer keeps. The bottom strip shows how far each measurement had to be rounded.
Try this: Press the "alias" preset and watch the gold line turn red and slow down. Then try "3 bits" and see the samples snap to the faint level lines.
A tone at frequency f and one at f + fs pass through exactly the same sample values, so the samples can't tell them apart. Any tone above half the sample rate folds back to the alias frequency. Converters put a low-pass filter in front of the sampler so nothing above fs/2 gets in.
Each extra bit halves the rounding step, which cuts the error power by 4, about 6.02 dB. The formula assumes the rounding error is spread evenly. At 1 or 2 bits, or when the tone lines up with the sample grid and repeats the same few values, the measured number drifts from it.
Young ears hear up to about 20,000 Hz. Twice that is 40 kHz, and 44.1 kHz leaves a margin for the filter that removes everything above 22,050 Hz. Sound travels about in air, so a 20 kHz wave is only long. CD audio uses 16 bits, about 98 dB between the loudest sine and the rounding hiss.
Multiply it out and stereo CD audio is 1,411,200 bit/s, roughly 10.6 MB per minute. The bits themselves are covered in C00 · Bits & bytes.
Your ears are not equally sensitive at every pitch. The threshold of hearing is the quietest level you can detect at each frequency, measured in decibels of sound pressure level (dB SPL, where 0 dB is roughly the faintest sound a young ear hears at 1 kHz). A loud tone also raises the threshold for nearby frequencies, so a quieter tone close to it disappears. That's simultaneous masking. An encoder doesn't spend bits on anything below the line. Drag the tones and watch which ones survive.
How to read this: Each stick is one tone. Anything under the amber line (the shaded zone) is drowned out or too quiet to hear, so the encoder spends no bits on it.
Try this: Drag the tall 1 kHz tone down and watch its neighbors turn green as the amber hump shrinks.
| Tone | Level | Threshold | Margin | Status | Bits |
|---|
The threshold in quiet is Terhardt's 1979 approximation to measured hearing data. It's a smooth fit to averages, so real ears vary by several dB. Bark is a frequency scale where each unit is one "critical band", a stretch of the inner ear that responds as one channel.
The masking curve here is a simplified two-slope model in the style used by perceptual coders. Masking reaches much farther upward in frequency than downward, and farther still when the masker is loud, which is why the upper slope gets shallower with level. Thresholds from several tones add as powers.
Each bit buys about 6 dB of signal-to-noise ratio (Lab 1). An encoder only needs the rounding noise to sit below the masking threshold, so a component just above the line needs a bit or two and a hidden one needs none. The "16 per tone" figure is a toy baseline for comparison.
A real MP3 encoder runs a psychoacoustic model on every frame of 1,152 samples, about 26 ms at 44.1 kHz, and hands the result to the quantizer in Lab 3.
Neighboring samples of music are similar, so storing every one of them in full wastes bits. Chop the signal into short blocks and rewrite each block as a sum of cosine waves. That's a transform, and the weights of the cosines are its coefficients. For tonal sound most of the energy lands in a few coefficients. Round them all to a coarse step and the small ones become zero. Long runs of zeros are cheap to store, and the decoder runs the transform backwards.
How to read this: In the map at the bottom, each column is one short block of sound and each row is one cosine ingredient. Green = a number worth storing; black = rounded to zero and stored almost for free.
Try this: Slide "Step size" to the right: the map goes black, and the ratio climbs. Press "Play decoded" at a big step to hear clicks at the block edges.
With this scaling the DCT is orthonormal: its inverse is its transpose, and the error energy in the coefficients equals the error energy in the sound. The bit count is the entropy of the rounded numbers, the size an ideal entropy coder would reach. It ignores side information such as step sizes, so it is an estimate.
MP3 (MPEG-1 Layer III) uses a close relative of this DCT. A filterbank first splits the sound into 32 frequency bands, then a modified DCT (MDCT) splits each band further, giving 576 frequency lines per half-frame. MDCT blocks overlap by half, and the overlap cancels the edge errors that cause the blocky clicks you can hear here with large steps. For sharp attacks the encoder switches to shorter blocks.
The step size for each group of lines comes from the psychoacoustic model in Lab 2, and the rounded numbers are packed with Huffman codes. The cosine view of sound is the subject of S02 · The Fourier transform.