The same filter, rounded twice

The filters field’s most useful result is not about a filter family at all. It is that two realisations of one response are two different objects, and that the difference is a matter of orders rather than of percentages: at the ripple peaks a one per cent component moves a buffered cascade by 0.162 dB and a doubly terminated ladder by 0.003 dB, with the cascade’s error growing as the 0.99 power of the tolerance and the ladder’s as the 2.00 power.

A digital filter has no components and no tolerances, and it would be reasonable to expect the question to disappear. It does not. A coefficient is stored in a finite number of bits, the rounding is a perturbation of exactly the same kind, and the answer comes out considerably harder than the analogue one — because a digital realisation can be made not merely inaccurate but unstable, by a rounding, from a design that is correct.

Why a coefficient is not a component #

The two perturbations look alike and are not, and the difference is what the whole essay turns on.

An analogue component perturbs one element. A resistor that is one per cent high changes one entry of the nodal matrix. Every pole moves a little, because the poles are the roots of the whole determinant, but the movement is bounded by how much that one element contributes to each.

A direct-form coefficient perturbs every pole at once, by construction. The denominator polynomial’s coefficients are the elementary symmetric functions of the poles: the second is minus their sum, the third is the sum of their pairwise products, and so on. There is no coefficient that belongs to one pole. Rounding any of them moves all of them.

A cascade’s coefficients are local. Each second-order section carries −2·Re(p) and |p|² for its own conjugate pair and knows nothing about the others, so rounding a section’s two numbers moves that pair and no other. The filter’s response is the product of the sections’ responses, so an error in one section is a bounded error in one factor.

That is the whole mechanism, and it predicts the shape of the measurement rather than merely agreeing with it: the cascade’s pole movement should be roughly independent of the order, and the direct form’s should grow rapidly with it.

What the measurement says #

One design — a Butterworth at 1 kHz, mapped to a 48 kHz clock with pre-warping, so its poles sit at a radius of about 0.9748. Both realisations, at five word lengths.

Two readings of that table are the essay.

The cascade’s movement barely depends on the order. 6.6 × 10⁻⁴ at eighth order and sixteen bits against 8.1 × 10⁻⁵ at fourth — a factor of eight for a doubling of the order, which is what a longer product of independently perturbed factors gives. Its largest pole radius stays at 0.9748 at every row, which is where it was designed: the poles are being nudged, not moved.

The direct form’s movement grows without any bound worth the name, and at eighth order it has already left the unit circle at sixteen bits. A radius of 1.4536 is not a filter with a disappointing passband. It is a recursion whose output grows by a factor of 1.45 every two samples — an oscillator that was designed as a low-pass filter, from a specification that is correct, by an implementation that rounded some numbers.

The gate checks the ordering at fifteen combinations of order and word length rather than checking the numbers, and separately checks the sharpest form of the claim: at eighth order the direct form is unstable at sixteen bits while the cascade is stable at eight.

Why the direct form’s coefficients are so badly conditioned #

The mechanism above says the direct form is worse. It does not by itself say how much worse, and the answer is the reason the effect is so violent at high order.

Consider what the coefficients actually are for this design. Eight poles clustered near z = 0.975 ± small, so the denominator is close to (1 − 0.975z⁻¹)⁸ — which expands to coefficients of size 1, 7.8, 26.6, 51.9, 63.2, 49.3, 24.0, 6.7, 0.8. The largest is sixty-three and the answer depends on differences between them.

Fixed-point storage scales by the largest coefficient, so the step is $63.2/2^{15} \approx 1.9\times10^{-3}$ at sixteen bits, and every coefficient — including the ones of size one — carries that absolute error. A relative error of two parts in a thousand on a coefficient whose contribution to a pole’s position is a difference between terms of size fifty is a large relative error on the difference.

That is the general form of ill-conditioning and this collection has met it in the same shape twice before. transferFunction in lib/response.js drops leading coefficients that are small relative to the largest, which is right, and means a network whose poles are 3.5 decades apart comes back first order — the s² coefficient is 10⁻¹⁷ of the constant. And the ladder field’s synthesis needs the same care: a prototype whose expansion ends with a constant remainder loses one of two elements if the alternation is misread.

The digital case is the same phenomenon with the roles of “large” and “small” set by the pole radius rather than by the frequency spread — and the closer the poles are to the unit circle, which is to say the lower the corner is relative to the sample rate, the worse it gets.

The one row that is not monotone, and why it is left in #

The cascade column at eighth order reads 6.6 × 10⁻⁴, 8.4 × 10⁻³, 8.4 × 10⁻³ at sixteen, twelve and ten bits — and at eight bits it is 1.2 × 10⁻¹ with a radius of exactly 1.00000. Two things there are worth not tidying.

The middle two are equal. Rounding to twelve bits and to ten bits put a section’s coefficient at the same rounded value, so the poles landed in the same place. That is not a defect of the measurement; it is what a coarse grid does, and a version of the table that smoothed it would be reporting a trend that the arithmetic does not have.

The eight-bit radius is exactly one. A pole at |p| = 1 is marginally stable — an undamped oscillation that neither grows nor decays — and it arrives here because the second coefficient of a section is |p|² and 0.9502 rounded to eight bits with a scale of two is 0.9531 … or, for one section, exactly 1. So the cascade at eight bits is not a filter with a slightly wrong response either. It is a filter with an oscillator in it, and it is the point at which the cascade’s own advantage runs out.

That the cascade fails at eight bits and the direct form fails at sixteen is the quantitative version of the essay’s claim, and it is a factor of two hundred and fifty-six in the coefficient resolution required — for one response, at one order, differing only in how the numbers were written down.

What this makes the design rule #

The rule that follows is short and is universally followed in practice, which makes it worth stating where it comes from rather than as a convention.

Realise a digital filter as a cascade of second-order sections, and never as a single high-order polynomial. The order above about four is where the direct form stops being usable at ordinary word lengths, and at eighth order it is unusable at any word length a processor offers.

There is a second-order consequence the measurement does not cover and it is worth naming: the order in which the sections are cascaded, and how the gain is distributed between them, both matter for overflow and round-off noise inside the filter — quantities distinct from the pole positions this essay measures. A section with a high-Q pole pair has a large internal gain near its resonance, so putting it first means its output overflows before the later sections have attenuated anything. Nothing here measures that, and it is carried forward.

What the coefficients look like when they are not badly conditioned #

The table above uses a corner at a forty-eighth of the sample rate, which is where the effect is severe. It is worth showing the other end, because it identifies what actually controls the answer and it is not the order.

Move the same eighth-order design’s corner to a quarter of the sample rate and its poles sit at a radius near 0.3 rather than 0.975. The denominator’s coefficients are then of size 1, 2.4, 2.7, 1.8, 0.8, 0.2, 0.04, 0.005, 0.0003 — a spread of three thousand rather than of sixty, but with the largest coefficients no longer dominated by cancellation, because the poles are far from the unit circle and the polynomial is not close to a repeated root.

The controlling quantity is how close the poles are to the unit circle, which is to say how low the corner is relative to the sample rate. A filter whose corner is a fiftieth of the clock has poles at radius 0.975 and a denominator close to (1 − 0.975z⁻¹)⁸, and a polynomial close to a repeated root is the canonical ill-conditioned object: its roots move as the nth root of a perturbation in its coefficients, so a relative error of 10⁻³ moves an eighth-order cluster’s roots by 10⁻³ raised to the power one-eighth, which is 0.42.

That exponent is the whole explanation for the violence of the direct form’s failure, and it is checkable against the table: 5.8 × 10⁻¹ of movement at sixteen bits, against a coefficient error of about 2 × 10⁻³, is an exponent of about 0.085 on that particular perturbation — the same order of magnitude as one eighth and in the right direction.

It also says what to do about it, which is the previous essay’s advice arriving from a different direction: oversampling makes a digital filter better conditioned as well as better shaped. A corner at a quarter of the sample rate rather than a fiftieth moves the poles away from the circle, away from the repeated-root cluster, and the direct form’s disadvantage shrinks with it. That is a fourth independent argument for a fast clock, and like the other three it is usually collected by accident.

Stability, and the field that owns it #

A pole outside the unit circle is an unstable filter, and this collection has a field whose whole business is stability — with machinery that is worth pointing at rather than duplicating.

The feedback field measures a loop’s stability by counting encirclements of the critical point, and it found something there that is directly relevant: a three-pole two-lead loop with a lead ratio of ten has two disjoint stable regions — below a gain of 979, and between 1.81 × 10⁶ and 3.36 × 10⁷ — so it is unstable both above and below an 18.6:1 window, and reducing its gain is what breaks it.

The digital question in this essay is simpler in one respect and harder in another. Simpler, because stability is a pole radius and needs no contour integral: root the denominator and look. Harder, because the thing being perturbed is not a physical parameter with a tolerance but a stored number whose error depends on the scaling, so there is no single perturbation size to sweep.

What the two share is the finding that stability is not a property a design has and keeps. It is a property of the realised object, and both fields arrived at the same place from opposite directions: an analogue loop that is stable at one gain and not at another, and a digital filter that is stable as a cascade and not as a polynomial, from one specification in each case.

What the measurement does not do #

Three limits, stated because the field is new and this is its last essay.

The rounding is uniform fixed-point. Coefficients are scaled by the largest and rounded to a word, which is the usual arrangement and is not the only one. Floating-point coefficients have a relative rather than an absolute error and behave differently; block floating-point differently again. The comparison between realisations survives all three, and the numbers are for one of them.

The poles are matched to their nearest neighbours when measuring how far they moved, which is the obvious construction and is ambiguous if two poles cross. At the word lengths where the direct form has left the circle the assignment is not meaningful anyway, which is why the radius rather than the movement is what the stability claim is made on.

Nothing here rounds the signal. Every number in this essay is about the coefficients. The arithmetic inside the filter is rounded too, at every multiply, and that produces a round-off noise floor and — in a fixed-point recursion — the possibility of a limit cycle, an oscillation of a few least significant bits that sustains itself on zero input. That is a boundary of exactly the kind this site exists to measure, it is bounded from below like the noise field’s, and it is not measured here. It is the clearest thing this field has left open.

What the whole field measured, and what it is a claim about #

This is the digital field’s last essay, so it is worth saying what the eleven of them together establish and — more usefully — what kind of claim each is.

Three of the boundaries are properties of the arithmetic and nothing else, true of any converter and any processor: half the sample rate, the sinc of the hold, and the warping of the map. None of them has a component value in it, none has a tolerance, and all three are computable exactly from a clock frequency.

Three are properties of the pair — a converter and the circuit it is attached to. The crossing with Johnson noise needs the source resistance; the anti-alias requirement needs the filter family and the bit count; the aperture’s cost in bits needs the input frequency. Each is a number a data sheet cannot state alone, and each restates the sentence the instruments field made first: an instrument is an element in the netlist.

Two are properties of the signal rather than of the equipment: the amplitude at which the quantisation error stops being noise, which depends on the levels the input crosses and not on how many the converter owns; and the crest factor that decides what 6.02N + 1.76 is a claim about.

And two are choices, which is what makes them the interesting end of the field. Where the quantisation noise sits is decided by whether there is a loop around the quantiser, and is worth six bits at an oversampling ratio of 128. How a filter’s coefficients are written down is decided by the implementer, and is worth a factor of two hundred and fifty-six in the resolution required — or, put the other way, is the difference between a filter and an oscillator.

Both of those choices have an essay each that measures what is inside them. One bit, and where the noise went is the first: a loop around the quantiser takes the in-band error down 8.99 decibels a doubling of the rate against oversampling’s 3.01, and at a ratio of 128 one bit is worth 9.18. And which section goes first is the second, one level below the choice this essay makes: having chosen a cascade of biquads, the twenty-four orders of four of them realise the identical transfer function at round-off floors running from 24 to 100 least significant bits, with thirteen of the twenty-four beaten on both counts by another arrangement. So the representation decision is not one decision but a nested pair, and the second one is free.

That last one is the reason this essay closes the field rather than the noise-shaping essay closing it. The collection’s premise is that no model is drawn without the frequency, amplitude or size at which it stops being true, and every boundary above obeys it. But a boundary that a designer puts there by choosing a representation is a stronger version of the same idea: the model did not stop being true because the physics ran out. It stopped being true because of how the numbers were stored, and there is a way of storing them for which it does not stop at all.