The average that is not N times quieter

The total that has no resistor in it found that the noise on a capacitor charged through any resistance is $\sqrt{kT/C}$ — 63.3 microvolts for a picofarad at room temperature — and that the resistance cancels exactly. The noise a clock does not make found the same number on a switched capacitor, independent of the clock. The sample that is subtracted removed it altogether by taking two samples of the same held charge and subtracting them.

Each of those was a statement about one sample. The standard move against a floor that cannot be subtracted is to take many samples and average them, and the standard arithmetic says that $N$ samples average to $\sqrt{kT/C}/\sqrt N$. That arithmetic assumes the samples are independent. Whether they are is decided by something none of those essays had to mention: how long the switch stays closed between one sample and the next.

What a sample remembers #

A switch closes on the capacitor, the capacitor tracks, the switch opens, the held voltage is read. While the switch is closed the capacitor’s noise is being refreshed through the switch’s resistance with the time constant $RC$: after a track time $T_t$ a fraction $\rho = e^{-T_t/RC}$ of the old noise voltage is still there, and the rest has been replaced by a fresh draw. The held sequence is

$$v_k = \rho\,v_{k-1} + \sqrt{1 - \rho^2}\;\sqrt{kT/C}\;w_k$$

with $w_k$ independent unit draws. Every sample still has variance exactly $kT/C$ — the refresh adds precisely what the decay removes, which is the same exact balance the noise a clock does not make found in a switched filter. What has changed is that consecutive samples share a fraction $\rho$ of each other, and the variance of their mean is

$$\frac{kT}{C\,N}\left[\frac{1+\rho}{1-\rho} - \frac{2\rho\,(1-\rho^N)}{N(1-\rho)^2}\right]$$

which is $kT/(CN)$ only when $\rho = 0$.

With one RC of tracking the correlation is 0.368. Sixteen samples are then worth 7.81 independent ones, and their average is quieter than a single sample by 2.8 rather than 4. At a quarter of an RC — the switch opened almost as soon as it closed — the correlation is 0.779 and sixteen samples are worth 2.63. With three RC they are worth 14.6, and with ten, all sixteen.

The departure is not a small correction at short track times. It is the difference between two bits of improvement and seven-tenths of one.

Why each sample is still exactly kT/C #

It is worth being sure of the first half of that before trusting the second, because the whole result rests on the variance of each sample being unchanged while the correlation between them grows.

While the switch is closed, the capacitor’s voltage is the output of a first-order filter driven by the switch resistance’s white noise, and its deviation from equilibrium decays with the time constant $RC$ while fresh noise is added. After a closure of length $T_t$, the old deviation has shrunk by $ho$, so its variance has shrunk by $ho^2$, and the fresh noise accumulated during the closure has variance $(kT/C)(1 - ho^2)$ — the part of the equilibrium the closure has had time to rebuild. The two add to $kT/C$ exactly, whatever $ho$ is. A closure of any length leaves the capacitor with the full equilibrium variance; what it does not leave is a new voltage.

That exactness is the same identity the noise a clock does not make found in a switched-capacitor filter, where the variance came out as $kT/C_h$ for every capacitor ratio because the recursion’s decay and its fresh noise combined in exactly this proportion. Here the recursion is simpler, and the lesson it adds is about what the identity hides: a sequence can have the right variance at every sample and still carry far less information than its length suggests.

What N samples are worth #

The quantity worth drawing is the effective number: $kT/C$ divided by the variance of the mean, the number of independent samples that would give the same result.

For a long average the bracket tends to $(1 + \rho)/(1 - \rho)$, so the average of $N$ is worth $N(1-\rho)/(1+\rho)$. The factor is a half at $\rho = 1/3$, which is 1.10 RC of tracking; for 64 samples, whose end effects still show, half-worth comes at 1.17 RC, and for sixteen at 1.08. Below about an RC most of each new sample is the old one.

The closed form is checked by the other route: a seeded march of the held sequence itself — the recursion above, run four thousand times — whose sample variance of the mean is required to match the formula within six of its own standard errors. It does, at four and at sixty-four samples, at every track time drawn. The march shares nothing with the formula but the recursion’s definition, which is the point of it.

The settling that already paid for it #

The marks on that figure are the part of the result that changes a design. A sampler exists to capture a signal, and the signal has to settle through the same switch resistance into the same capacitor: to half a least significant bit at $b$ bits takes $\ln 2^{b+1}$ time constants — 6.24 RC at eight bits, 9.01 at twelve and 11.78 at sixteen.

At every one of those track times the noise samples are already independent, to within a per cent at eight bits and to far better at twelve. So a sampler that re-acquires its signal every cycle — which is what a converter’s sample-and-hold does — has necessarily waited long enough for its noise, and its averages behave as the textbook says. The correlation is not a hazard of converters that settle properly. It belongs to a different habit.

The habit that meets it #

The habit is sampling a slow input quickly. A signal that is steady — a bridge’s output, a thermocouple, a reference — has already settled on the capacitor from the samples before, and nothing about the signal requires the switch to stay closed for nine time constants. So the track time is cut to the minimum the switch driver allows, the sample rate goes up by ten, and the extra samples are averaged, on the expectation that a tenfold rate is a tenfold average and a factor of 3.2 in noise.

It is not. The signal needed no track time, but the noise did: kT/C is refreshed only as fast as the switch’s RC allows, however steady the input is. Cutting the track time from nine RC to a quarter of one raises the sample count thirty-six times over the same period, and — as the next figure shows — raises the independent information gathered in that period by a factor of only four and a half, with each individual sample worth an eighth of one.

The ceiling on the rate #

For a fixed averaging time rather than a fixed count, the question is how many independent samples the time can contain. Taken slowly, one per track time. Taken faster, each is worth less, and the two effects meet at a limit.

The ceiling is one independent sample per $2RC$, and it is not a property of sampling at all. As the track time goes to nothing the samples become a continuous reading of the capacitor’s voltage, and the continuous average of an RC-filtered noise over a time $T$ has variance $(kT/C)\cdot 2RC/T$ — the noise density times the average’s own noise bandwidth, $1/(2T)$, which the filter an average is derives for any window. That is exactly $T/(2RC)$ independent samples’ worth.

So the design statement is short. An average over a time $T$ can never beat $kT/C$ divided by $T/(2RC)$, however many samples it takes. Sampling once every RC reaches 92 per cent of that ceiling. Sampling ten times faster reaches 99 per cent of it and costs ten times the conversions. Sampling ten times slower — once per ten RC — reaches a fifth of it, because now the samples are independent but there are too few of them.

The time that counts is the time the switch is closed #

One qualification makes the ceiling sharper and moves it in the direction a converter designer will not like. The refresh happens only while the switch is closed. While it is open — which is when a converter holds the sample and converts it — the capacitor is isolated, its noise voltage is frozen, and nothing about it is renewed. The correlation between consecutive samples is set by the track time alone, and the conversion time in between counts for nothing.

So the ceiling of one independent sample per $2RC$ is per $2RC$ of tracking. A converter that tracks for a tenth of each cycle and converts for the rest gathers, at best, one independent sample per twenty RC of wall-clock time, and the remaining nine-tenths of its time buys the conversion and nothing towards the average. A slow input sampled by a converter with a short acquisition phase is therefore limited twice: by the track time making each sample correlated with the last, and by the conversion time making the tracking a small share of the whole.

The numbers are worth putting into units. A picofarad behind a kilohm of switch resistance has an RC of a nanosecond, and the ceiling is five hundred independent samples per microsecond of tracking. A converter at a million samples a second with a hundred nanoseconds of acquisition each tracks for a tenth of a microsecond per microsecond, so it can gather at most fifty independent samples of kT/C per microsecond whatever it does — and it gathers one, because it takes one sample. Averaging a thousand of its samples, a millisecond’s worth, reaches $63.3/\sqrt{1000} = 2.0$ µV because each of its samples is independent: a hundred nanoseconds is a hundred RC. The correlation only bites when the acquisition is cut towards a few RC to raise the rate, which is exactly the habit described above.

What the noise bandwidth was saying all along #

The ceiling ties this back to a result that did not look like it was about sampling. The total that has no resistor in it found kT/C independent of the resistance because a larger resistance is noisier in exact proportion to how much narrower it makes the band. That same narrower band is what makes consecutive samples correlated: the capacitor’s noise has a bandwidth of $1/(4RC)$, and samples taken faster than that bandwidth are oversampling a signal that has no more information in it.

Averaging is a filter, and the average’s variance is the density multiplied by the narrower of the two bandwidths — the RC’s own, or the average’s $1/(2T)$. While the average is short, the RC’s bandwidth is the wider and the average does the limiting; the textbook $1/N$ holds only while the samples are sparse enough that the average’s bandwidth is set by $N$ rather than by $T$. The bandwidth noise sees is the arithmetic of bandwidths, and here it is the whole of the result: the average cannot see less noise than its own bandwidth passes.

Where the correlation is useful instead #

A correlation between samples is not always a loss. The sample that is subtracted exploits the extreme case: when the switch stays open between two reads of the same held charge, the correlation is exactly one, and the difference of the two reads removes kT/C entirely. That is the same recursion with $\rho = 1$ and a difference instead of a sum.

The two results are the two ends of one quantity. A difference of correlated samples is quieter the more correlated they are; a sum is quieter the less. A design that averages wants its switch closed long between samples, and a design that subtracts wants it not to close at all. Anything in between gets some of each, and the correlation coefficient $\rho$ is the one number that says how much.

What an averaging converter should do with it #

Three rules follow, and each is a number rather than a habit.

Track for an RC or more if the samples are to be averaged. Below an RC each sample is mostly the last one; above three it is nearly all new. Between them the effective count follows the second figure, and a design can read off what its acquisition time buys. The rule that a converter’s acquisition should settle its signal already puts it far past this, so the rule only needs stating for a converter that shortens its acquisition on purpose.

Count bits by the effective number, not the actual. Averaging is usually priced at half a bit per doubling of the count — six decibels a bit is the conversion — and that price is paid in effective samples. At a quarter of an RC a doubling of the actual count is a doubling of the effective count only once the average is long enough for the end effects to vanish, and each effective sample costs eight actual ones.

Remember that the kT/C noise is also the dither. A converter averaging a steady input relies on noise to spread the input across its codes, since without it every sample returns the same code and the average cannot resolve anything finer than a step. The dither that is a decision sizes that dither; on a converter whose own sampling noise is the only dither, correlated samples are also correlated dither, and the codes the average visits are fewer than the count suggests. Whether the loss in resolution is then worse than the loss in noise is not measured here; that the same correlation causes both is.

Where the closed form meets the march #

The closed form and a seeded march agree at four and at sixty-four samples, at every track time drawn, within six standard errors of the march’s own estimate — four thousand trials, so a one-per-cent test.

No average is worth more than its count, and at ten RC sixteen samples are worth sixteen to the precision printed.

At the settling time for 8, 12 and 16 bits, 64 samples are required to be worth at least 99 per cent of 64, which is the statement that a sampler which settles its signal has settled its noise.

And the long-average rate is required never to pass one sample per 2RC, and to approach it within a per cent as the track time shrinks to a hundredth of an RC.

A clean switch and a capacitor with no flicker #

The switch is a resistor while closed and nothing while open. A real switch’s resistance depends on the voltage, and its opening injects charge; both matter to the held value and neither changes the correlation arithmetic, which is about how much of the old noise survives a closure.

The noise is only the capacitor’s. An amplifier driving the capacitor adds its own noise, sampled, and the amplifier inside the sample finds that it folds into the band once per time constant of settling. Its samples are correlated by the same track time, with the amplifier’s bandwidth in place of the switch’s.

And nothing slow is present. A flicker component makes consecutive samples correlated at every track time, and no average removes it — the corner where averaging stops working is that limit, arriving from the low-frequency end where this one arrives from the high.

Still open: the integrator, the amplifier’s share, and the optimum rate with a cost #

Averaging by integration instead. A switched-capacitor integrator adds each sample’s charge into a feedback capacitor, and the samples it adds are independent — its sampling capacitor is reset by each closure — but each cycle has two switch phases and so two kT/C draws. Whether an integrating average pays twice per sample and gains full independence, and what limits how long it can integrate, is the natural comparison.

The amplifier’s noise, correlated the same way. Driving the capacitor from an amplifier adds a second noise with its own bandwidth. Averaged at a short track time it is correlated by the amplifier’s bandwidth rather than the switch’s, and the ceiling on the rate is then set by whichever of the two is narrower.

A rate with a price. Every conversion costs energy. With the variance falling as the effective count and the energy rising as the actual count, there is a track time that minimises the energy for a given noise, and by the ceiling above it should sit near an RC — a number a low-power converter would want before it chose its clock.