An exact answer to a different question

The previous essay showed two signals producing one set of samples and stopped there. That is the whole identity, but it leaves a question a reader is entitled to ask: if the samples are ambiguous, what exactly does the machine that turns them back into a signal produce? Something averaged? Something distorted? Something with an obvious artefact in it that a careful engineer would notice?

None of those. It produces the alias, to the same accuracy it produces the input with when the input is inside the band — and that is a stronger and less comfortable statement than “aliasing degrades the signal”.

The sum, and what it is being asked #

The reconstruction the sampling theorem prescribes is the Whittaker–Shannon sum: place a sinc function at every sample instant, scale each by its sample, and add them.

$$x(t) = \sum_k x[k]\,\operatorname{sinc}\!\left(\pi\left(t f_\mathrm{s} - k\right)\right)$$

It has the property the theorem needs — the sinc is one at its own sample instant and zero at every other, so the sum passes exactly through every sample — and it has one more that matters here: among all signals through those samples, it is the one containing nothing above half the rate. That is what makes it the reconstruction rather than a curve through the points.

So the sum answers a specific question, and the question is not “what was the input”. It is: of the infinitely many signals through these samples, which one is inside the band? Below half the rate the answer is the input, because the input was inside the band. Above half the rate the answer is the image, because the image is inside the band and the input is not.

Nothing goes wrong. The sum does its job, correctly, and its job was never to recover the input.

Two references, one measurement #

The measurement lib/sample.js performs is deliberately blunt. It samples a sinusoid, evaluates the sum at four hundred points across the record, and takes the worst departure from a stated function. Then it does it twice: once against the input, once against the image.

Three numbers come out, and it is worth putting them in one place.

The first two are the same kind of number — within a factor of four of each other, and the factor is the input frequency rather than the boundary, since a tone near half the rate has fewer samples per cycle for the sum to work with. Neither is zero, and neither is zero for the same reason: the sum is truncated at sixty-four samples either side of the point being evaluated, and a sinc falls only as 1/k, so the tail that is thrown away is of order 1/(π × 64). That is a property of this implementation and not of the theorem, and the site’s habit is to quote it rather than to hide it inside a tolerance.

The third number is not that kind of number at all. Two is the whole amplitude of the signal, doubled — the reconstruction and the input are in antiphase at the worst point, so the departure is the sum of two amplitudes rather than the difference of two similar things. The ratio between the second and third is 1.26 × 10³, and the gate requires it to exceed two hundred so that a change which quietly blurred the distinction would fail rather than pass.

Why the guard band is excluded, and what it costs to say so #

The sum needs samples on both sides of the point it is evaluating, and near the ends of a finite record it does not have them. Evaluating there measures the truncation of a one-sided sum, which is much worse than the truncation of a two-sided one and has nothing to do with the sampling.

So reconstructionError excludes a guard of sixty-eight samples at each end and says that it does. This is the kind of exclusion that is easy to make quietly and worth making loudly, because the numbers it protects are the ones the whole comparison rests on. Without the guard both errors would be an order of magnitude larger and would still be equal to each other, so the argument would survive while every number in it became a measurement of the window rather than of the sampling.

The general form of that trap is one this collection has met twice already. The noise field’s two routes to a noise voltage disagreed by two to seven per cent until two discretisation errors were found, and widening the tolerance would have hidden both. A number that is not what it claims to be usually still moves the right way; that is exactly what makes it durable.

How slowly the sum converges, measured #

The truncation was described above as “of order 1/(π × 64)”. That is an estimate from the shape of the sinc, and this collection’s rule is that an estimate of that kind gets measured rather than quoted — the same rule that turned the trapezoidal rule’s claimed second order into a measured factor of 4.00 per halving of the step and the lumped ladder’s approach to the wave limit into a measured exponent of −0.54.

Run the same reconstruction at six truncation lengths, on the same samples, against the same reference:

Fitted across all six, the error falls as the −1.02 power of the number of terms, which is the 1/k tail of the sinc read directly off the measurement. That is a slow convergence by the standards of the rest of this site: the trapezoidal rule gains two digits for every factor of ten in the step, and this gains one for every factor of ten in the effort.

The middle of the table is worth not smoothing over. At thirty-two terms the error is worse than at sixteen, which is not noise and not a defect. A sinc’s tail alternates in sign, so a truncation that stops on a favourable half-cycle beats one that stops on an unfavourable one, and the envelope rather than the sequence is what falls as 1/k. A version of this measurement that had reported only the two ends would have shown a tidy factor of thirty-seven across five doublings and hidden a real property of the method.

The practical consequence is the reason the ideal reconstruction is a measuring instrument here and not an implementation. Reaching sixteen-bit accuracy with this sum needs of order sixty-five thousand terms per output point, which is why nothing does it — and why the crude reconstruction below is what every converter actually performs.

What a converter actually does instead #

No converter evaluates that sum. A digital-to-analogue converter holds each sample for a clock period, which is a much cruder reconstruction, and the difference between the crude one and the ideal one is itself measurable and is an essay further along this field.

It is worth being clear that this does not rescue anything. The hold is a worse reconstruction than the sum, not a differently-behaved one: it produces the same in-band signal with a droop across it, plus images around every multiple of the clock that a reconstruction filter then has to remove. If the in-band signal is the alias, the hold reproduces the alias with a droop on it.

The ideal sum is therefore the right object to measure the boundary with, precisely because it is the best case. Anything a real system does is a degraded version of it, and a degraded version of an exact answer to the wrong question is still an answer to the wrong question.

Reading the curve, and one thing it does not say #

The figure draws both errors as functions of the input frequency, and two features of the shape are worth naming.

The upper curve rises before the boundary. Approaching 5 kHz from below, the error against the input grows from about 5 × 10⁻³ at the bottom of the axis towards a tenth. That is not aliasing beginning early; it is the truncated sum having a harder job, because at a frequency approaching half the rate there are barely two samples per cycle and the sinc tails carry more of the answer. Refusing to explain that as “aliasing setting in gradually” matters, because it is the one place in this essay where a real effect could be misread as a soft boundary. The boundary is not soft. The measurement of it, with sixty-four terms, has a shoulder.

The lower curve is flat across the boundary. It measures the error against whichever frequency is inside the band, which below 5 kHz is the input and above it is the image — the same quantity, read against a reference that swaps at the boundary. Its flatness is the point of the whole figure: the sum’s performance does not change at all as the input crosses. What changes is what it is performing on.

The filter after the converter, which has the same trade as the one before #

A held output carries the in-band signal with a droop on it and a copy of the whole band around every multiple of the clock. Removing those copies is another analogue filter, and it is the mirror image of the anti-alias filter that the next essay in this field measures — same components, same families, same trade between skirt and delay, run backwards.

It also inherits the same arithmetic. The first image band starts at fₛ minus the highest wanted frequency, so the filter has from the top of the band to there in which to fall, and how much falling it can do in that span is a property of its family and order rather than of the converter.

The two are not quite symmetric, and the asymmetry is in the droop. An anti-alias filter’s passband ripple is its own; a reconstruction filter’s passband must accommodate the hold’s sinc as well, which is 3.92 dB of tilt across the band at the extreme and cannot be designed away by choosing a better family. The usual answer is to compensate for it digitally before the converter — a filter whose passband rises to meet a droop it cannot see — which is one of the places where the analogue and the digital halves of a signal chain stop being separable.

Where the boundary is not a boundary #

Two cases fall outside everything above, and both are worth stating so that the claim is not read wider than it is.

A signal with no content above half the rate is fully recovered, and the theorem is not approximate about it. That is the positive half, and this essay has spent its words on the negative half because the positive half is the one already believed. The measurement supports it: below the boundary the reconstruction is within seven parts in a thousand of the input at every frequency tried, and the residual is the truncation rather than the theorem.

A signal deliberately placed in an image band is recovered too, which is the technique the previous essay described. Nothing in the sum objects; it returns the band-limited member of the family, and if a designer arranged for that member to be the one carrying the information then the reconstruction is exactly right. The essay’s title is not a complaint about the sum. It is a description of what a reconstruction is: a choice of one member of a family, made by a rule that knows nothing about which member anybody wanted.

Why this essay measures a worst error rather than a spectrum #

There is an obvious alternative way to make the same argument: transform the reconstruction and show a line at 1 kHz where a line at 9 kHz was expected. It would be a shorter essay and a worse measurement, for a reason worth stating because it applies well beyond this figure.

A spectrum answers “how much of each frequency is present”, and the claim being made here is stronger than that. It is that the reconstruction is a particular function of time, matching one reference and not another, at every instant rather than on average. A worst-case departure across four hundred points supports that; a spectral line does not, because two signals can share a spectrum and differ in phase — which is exactly the case here, since the image arrives with its phase reversed and a magnitude spectrum would show the two as identical.

The site has made this choice before and for the same reason. The step response is computed twice, once from residues and once by walking the network forward in time, rather than being inferred from a transfer function; and the three-phase neutral is computed both from the solve and from the zero-sequence component, because two routes that share only their input are worth more than one route reported precisely. A time-domain comparison against two references is the same habit applied to a sum instead of to a network.

What this leaves for the rest of the field #

The boundary is now measured from both directions and the machinery for it is checked. Everything after this essay is about the two things a converter does that the sampling theorem says nothing about at all.

It replaces each sample with one of a finite set of numbers, which is a boundary from below in amplitude. The floor a converter sets measures where that floor sits against the circuit in front of it, and finds the two crossing at 18.80 bits for a kilohm source in a hundred kilohertz — past which every further bit buys a more precise measurement of thermal noise. When a floor stops being a floor then measures the amplitude at which it stops behaving like a floor at all: the share of the error’s power sitting in harmonics of the input rises from 0.29 per cent at 230 levels to 76 at 1.3, with the total unchanged, so what fails is the description rather than the quantity.

And it takes each sample at an instant that is not quite the right instant, which is a boundary in time whose cost is proportional to the slope of the signal and therefore to its frequency. A picosecond, read as bits turns that into the units a designer uses — ten picoseconds holds sixteen bits up to 199 kHz and twelve bits up to 3.18 MHz, falling at exactly twenty decibels a decade — and a floor, or a line then shows that the same picoseconds can put the resulting error in two entirely different places, twenty-eight decibels apart, with nothing in the specification to distinguish them.

Neither is in the theorem, both are in every converter, and both are measurable in the way everything on this site is measurable: build the thing, run it, and read the answer back out. That is worth saying plainly because the theorem’s own exactness invites the opposite habit — a result proved rather than measured tends to be quoted rather than bounded, and a converter is the object where the proof and the part have the least in common.

There is a fourth thing, and it is the one this essay is closest to. The staircase on the way out measures what happens when the reconstruction is not the ideal sum at all but the thing a converter actually does — hold each sample for a whole clock period, which is a convolution with a rectangle and therefore a multiplication by a sinc: 0.14 dB down at a tenth of the sample rate, 0.91 at a quarter and 3.92 at half, which is exactly $20\log(2/\pi)$. Nobody chose that droop, and it is larger than the passband ripple of every filter this site has designed. So the exactness measured on this page is the property of an interpolation nothing implements, and the error of the one that is implemented is deterministic, known in closed form, and larger than most of the errors anybody argues about.