Concept

Numerical error — where it appears

The difference between what an algorithm returns and what the equations it approximates say, set by its step, its rule and the arithmetic's precision. It is not predicted by a matrix's condition number: a bridge near balance loses every digit on a matrix in perfect health, and a high-gain loop loses none on one that is nearly singular.

Named by 18 essays across 8 fields — each of them below, with the objects they name alongside it.

One step response, computed twice: from the poles, and by walking the network forward. A damping ratio of 0.22, so the overshoot is 49.2%. The two curves are drawn on top of each other; the panel below is the difference between them, which is the trapezoidal rule's error at 500 steps and reaches 1.70e-3 V.

One step, computed twice

A step response from the poles is exact. The same step walked forward in time is not, and the difference between them is the trapezoidal rule's own error rather than anything about the circuit. It falls by a factor of four every time the step is halved, which is a claim about a method and can be watched.

transients · Step response
A network the solver will answer, and should not be asked, into 0.01 Ω. computed by solving, not by drawing at 61 spreads. A hundredth of an ohm either side of a wire whose resistance is made smaller and smaller, into 0.01 Ω — an element written where the right answer is no element at all. The solution stays exact: 0.499999998422607 at the last spread before the refusal, against 0.500000000000000. What grows is the current-law residual, as the 0.96 power of the spread. The solve is refused at a spread of 2.5e+8, where the smallest pivot falls under 1e-12; the residual tolerance of 1e-7 would have been reached at about 3.3e+9. Two guards written for unrelated reasons, arriving within a decade of one another.

The answer that is perfect and absurd

A network with a wire written into it as a small resistance returns the right node voltage to fifteen figures, passes both verifications with a residual of two parts in ten to the sixteenth, and reports two hundred thousand amperes. The solver refuses it one decade further on, and by then it has been answering for eight decades.

networks · Singular-network
A sum that is exact, and the bandwidth estimate that is not. computed by solving, not by drawing, at 28 spreads of the three capacitor values in a resistor chain. The sum of the open-circuit time constants — each capacitor's own value times the resistance seen at its terminals with the other two removed — is 600.00 µs here, and it equals the ratio of the first two coefficients of the denominator to 2.0e-9 and the sum of the negated reciprocal poles to 2.0e-9. That much is a theorem. What is an estimate is the bandwidth: one over 2πΣτ gives 265.3 Hz against a measured 309.2 Hz, low by 14.2%. It is low at every spread on the axis — the estimate is never optimistic — and comes within ten per cent only once one of the three time constants is 7.48 times the others.

A sum that is exact, and the estimate that is not

Add each capacitor's value times the resistance seen at its own terminals with the others removed, and the total is the ratio of the first two coefficients of the denominator polynomial — a theorem, holding to a part in a billion at every spread tested. Divide one by two pi times it and you have a bandwidth estimate that is 14 per cent low with three equal capacitors and never once optimistic. Two settings of the slider have the same three time constants and bandwidths two per cent apart, which is why the sum can never be more than an estimate.

transients · Open circuit time constants
Fourteen decades of imbalance, fourteen digits gone, and a matrix in perfect health. computed by solving, not by drawing. A Wheatstone bridge walked towards balance, with the relative error of the solved output against a closed form that cannot lose digits. The condition number of the nodal matrix is 505.0 at every imbalance and the smallest pivot is 2.0e-3 of the matrix norm — orders above the 1e-12 at which this solver refuses to answer at all. Neither number moves, and the answer still loses one digit per decade of imbalance, reaching 33% at δ = 1e-15. The bound drawn over it is the round-off divided by the imbalance, which the measurement stays under at every point. The third curve is the same closed form written as ½ − 1/(2+δ) — algebraically identical, and it loses its digits at the same rate, which is where the loss lives: in the subtraction of two nearly equal numbers, not in the matrix.

The matrix that is ill, and the answer that is not

Every other essay here treats the solve as exact, and it is not. A bridge walked towards balance loses one digit per decade of imbalance and has none left at a part in 10¹⁵ — on a matrix whose condition number never moves and whose smallest pivot stays four orders above the threshold this solver refuses at. A feedback amplifier does the opposite: the solver declines to answer at a gain of 10⁹, one decade after it returned an answer that was exact to the last bit.

networks · Conditioning
A difference quotient is best at a step of 1e-5, and is 1e+6 times worse at 10⁻¹¹. computed by solving, not by drawing. The worst disagreement between the adjoint network's derivatives and a central difference quotient of the same quantities, against the fractional step the quotient is taken with, on a 7-element ladder at 1000 Hz. The curve has a minimum because two errors pull opposite ways: the curvature the quotient neglects falls as the square of the step, and the digits its subtraction destroys rise as one over the step. The best it reaches is 1.4e-9, against the 2.3e-11 that the two-thirds power of the machine epsilon predicts. The exact route costs 2 solves against 15, and has neither error term.

Every derivative, and the one that is zero

How much does this response move if that capacitor is one per cent out? A difference quotient answers it one component at a time, in two solves each, and its best possible accuracy is four parts in a hundred million. Transposing the matrix and solving once more answers it for every component at once, exactly. Pointed at a claim this collection has made since its ladder essay and never tested directly — that a doubly terminated ladder's response is stationary in every element at its passband maxima — it returns two parts in ten billion, where the cascade realising the identical response returns 0.72.

networks · Sensitivity
A comparator's delay, at a divider ratio of 0.50. Each nanosecond of comparator delay adds 2.650 nanoseconds to the period — measured as a slope between two delays, both exact multiples of the marching step — against the 2.667 that 4/(1+β) gives and the 2 that counting the delay twice gives. A 50 ns comparator therefore holds the frequency to one per cent only below 75.0 kHz.

The period a delay lengthens

Feed a comparator's output back through a resistor to the capacitor on its own input and the two thresholds stop defending a decision and start setting a period. The closed form is two RC times the log of one plus beta over one minus beta, and a marched circuit recovers it as the step shortens. A comparator that responds fifty nanoseconds late does not add fifty nanoseconds to each half cycle: it adds 4/(1+beta) times the delay to the period, 2.65 here against the 2 that counting it twice gives, because during the delay the capacitor keeps going the way it was going.

applied · Hysteresis
Sixteen more digits move the boundary by sixteen decades and leave it exactly where it was. computed by solving, not by drawing. The rung below's bridge, walked towards balance and solved twice: once in double precision and once with a pair of doubles carrying about 31 decimal digits, against a closed form that cannot lose any. The 33 per cent error at an imbalance of 10⁻¹⁵ becomes 7.0e-18 — so that loss was the arithmetic's and not the network's, which is what the rung below could not say. Each arithmetic's error is its own round-off divided by the imbalance, drawn as the two straight lines, so the second boundary is the first one moved by exactly the extra digits. The condition number is 505 in both cases and at every point, which is the diagnostic being blind twice over.

The digits the arithmetic did not have

The rung below bounded this site's own arithmetic and found two boundaries it could not attribute: a bridge with no correct figures left at an imbalance of 10⁻¹⁵, and a filter synthesis that stalls at order 14. An ill-conditioned problem stays ill-conditioned however many digits are used, and a well-conditioned one computed badly gets better — so adding digits is the experiment that tells them apart. The bridge's loss is entirely the arithmetic's. The synthesis's is mostly the data's, and doubling the digits makes it worse.

networks · Conditioning
The floor rises as the 1.06 power of the order, and the arithmetic gives out first. computed by solving, not by drawing. The least departure a doubly terminated ladder can achieve at any impedance level, against order — the floor, which is what is left when the tolerance the band is drawn at is taken away. It rises from 0.0087 dB at order three to 0.0389 dB at order 13, as the 1.06 power of the order, so the order at which it reaches the 0.1 dB the band is drawn at is near 32. Drawn over it are the same networks with one imperfection removed at a time, and they do not add: the winding loss alone leaves a larger departure than the complete realisation does, because the track resistance is in series with the load and lifts the passband exactly where the winding loss droops it. The stray capacitance contributes nothing at all, since the level that minimises the departure is a few ohms. What ends the sweep is neither: the continued-fraction synthesis loses its leading coefficients to cancellation and stalls at order 9 for a Butterworth and 14 for a Chebyshev.

The floor that outlives the arithmetic

A band of impedance levels is what a tolerance allows; a floor is what a structure has. Measured at six orders, a doubly terminated ladder's floor rises as the 1.06 power of the order and would not reach a tenth of a decibel until order thirty-two — an order nobody builds. What stops the sweep is neither the structure nor the parasitics: the continued fraction that turns a reflection polynomial into element values loses its leading coefficients to cancellation and stalls at order nine for a Butterworth and fourteen for a Chebyshev.

filters · Impedance scaling
5 sections, equal ripple, and the band that is 134% rather than 97. computed by solving, not by drawing. The repaired five-section equal-ripple design over the band it was designed for. The horizontal rule is the 0.1 the specification allows and the 4 interior peaks sit on it, level to 8.5e-4 per cent — which is the condition for a minimax solution and is now checked rather than assumed. The dots are the eighty-one frequencies the objective used to be evaluated at: the worst of them is 0.09999998 and the worst of the design over the whole band is 0.10013858, so an optimiser shown only the dots drove them down to the specification and left the true peaks 13.9 parts in ten thousand above it. That is nothing until something downstream is a threshold, and the band measurement was one: it reported 97.34 per cent for a design that holds 134.04.

The number that was wrong

The rung below printed 97.3 per cent of band for a five-section transformer where the answer is 134, said in its own text that the figure was wrong, and blamed a search that had converged to eight digits. The search was fine. The objective was the worst of a grid rather than the worst of a band, the band was then measured by bisecting a function that crosses its threshold five times, and the assertion guarding all of it passed — because 97.3 is still more than 92.6.

lines · Matching
The sensitivity of a pole against the room it has. computed by solving, not by drawing. A series R–L–C whose damping is walked from 0.3 to 0.999999, which slides its two poles together along a straight line and changes nothing else. The exact derivative of a pole with respect to the capacitor climbs from 0.5241 to 353.6 as the gap between them falls from 19078 to 28.28 radians a second. The fitted exponent over the closest four is -1.0000, and the product of the two is the natural frequency itself — 9999.6894 against 9999.6894, at every damping drawn and not merely in the limit, which a closed form gives and this computation never sees. The resistor's curve runs at 2ζ times the capacitor's — below it at 0.3 and at twice it by the time the poles have met — and the inductor's lies exactly under the capacitor's throughout.

The gap a derivative needs

The derivative of a pole is exact and has no step size in it, and beside the formula sits a sentence nobody had measured: it divides by a quantity that vanishes when two poles meet. Driven together, the sensitivity climbs as the reciprocal of the gap — fitted exponent −1.0000, the product a constant 1.00000 times the natural frequency — while the largest change it still describes falls as the gap *squared*. A one per cent capacitor is outside first order once the poles are 3194 radians a second apart, which is an ordinary critically damped design.

transients · Poles
Fed nothing, an eighth-order cascade sits 88 least significant bits from zero. computed by solving, not by drawing. Where each section of a 12-bit rounded cascade settles with zero input and a seeded state. Nothing decays to zero: a constant state survives whenever rounding returns it to itself, which needs only |y·(1 + a₁ + a₂)| ≤ q/2, and that denominator is small precisely because the corner is far below the sample rate. Each section's own band is 30, 31, 32, 33 least significant bits and the settled offsets are -27, -50, -70, -88 — accumulating, because a section's dead-band output is the next section's input and the next section passes direct current. In volts the offset halves with every bit added; in least significant bits it does not move at all.

Zero in, and not zero out

The rung below rounds a filter's coefficients and marches it in double precision, and its own list of what it did not do names the other half: the products are rounded too. Put that in and a twelve-bit eighth-order cascade fed nothing at all settles two per cent of full scale away from zero and stays there — and the offset does not shrink with the word length, it grows as the square of how far the corner sits below the sample rate, reaching eighteen per cent at a hundred and twenty-eight times.

digital · Digital realisation
Inside the passband the matrix is worst at the edge, and at no ripple peak. computed by solving, not by drawing. The condition number of a 5th-order Chebyshev filter's nodal matrix at every frequency from a hundredth of its cutoff to ten times it. It is 4.004e+3 at direct current, rises to 1.5868e+7 at 964.0 Hz — located by golden section rather than read off the sweep — falls to 4.970e+6 at 1520 Hz and rises again through the stopband. The maximum is above every ripple peak (the highest is at 897.9 Hz) and within 3.6 per cent of the half-power frequency: the worst-conditioned place in the passband is where the filter stops passing and starts blocking, which is a place the response curve has no feature at. It is a local maximum: above the band κ climbs again, reaching 2.702e+7 at ten times the cutoff, because the susceptances in the matrix grow with the frequency and κ counts them. Nothing about any of these numbers says whether a digit is actually lost anywhere.

Where the matrix is worst, and where the answer is not

When a solve stops being exact has been answered with two networks at direct current. Swept along the frequency axis, a resistive chain's nodal matrix has a condition number of 4.00×10³ at direct current and 1.59×10⁷ at 964 Hz — a maximum at the band edge, at no ripple peak and at no feature the response has. It predicts nothing. The solution vector is right to three units of round-off everywhere, the response taken out of it loses four decades into the stopband, and the same filter written at 400 kΩ instead of 10 Ω has a condition number 1.4×10⁹ times larger and returns the same twelve digits.

networks · Conditioning
Twenty-four orderings, and 11 of them are choices. computed by solving, not by drawing. Every ordering of the four sections of an eighth-order 0.5 dB Chebyshev, at 16 bits, drawn against the two things an ordering decides. The horizontal axis is the round-off floor the arrangement adds — 24.2 least significant bits at best and 99.8 at worst. The vertical axis is the largest value any section's output reaches, which is what decides whether a word overflows: 0.088 of full scale at best and 0.699 at worst, a range of 18.0 decibels. 13 of the twenty-four are beaten on both counts by another ordering and are simply mistakes; the 11 on the lower-left frontier are the actual choices, and no one of them is best.

Which section goes first

A cascade of four biquads can be assembled in twenty-four orders, all of which realise exactly the same transfer function. They do not cost the same: the round-off floor runs from 24 to 100 least significant bits and the largest value any section reaches runs over eighteen decibels — and the two go opposite ways, so eleven of the twenty-four are genuine choices and thirteen are beaten on both counts by another arrangement.

digital · Digital realisation
Loop gain of a two-pole amplifier closed for a gain of 100. Unity loop gain at 5.73 kHz, where 35.0° of phase remains before −180°. The phase never reaches −180° at any frequency, so there is no gain margin to quote: 2 poles contribute at most 180° and the last of it arrives only at infinity.

The loop that never crosses

Every loop this field draws carries two margins, and one of them is not always a number. Take the third pole out of the standard loop and its crossover moves by 2.45 parts per million and its phase margin by 0.164 degrees — the third pole's own arctangent there, to five decimal places — while its gain margin goes from 46.06 decibels to no number at all. The phase reaches −180° only where the magnitude has already reached −62 decibels, and the two instruments that are supposed to notice report 3.19 × 10⁻⁶ either way.

feedback · Loop gain
The circuit does not care which node is called zero, and the matrix does. computed by solving, not by drawing. The condition number of the nodal matrix for a 12-section chain, against which of its nodes was taken as the reference. The network, its elements and its physics are identical in every case — only a label has moved — and every branch voltage and branch current comes back the same to 1.1e-13. The condition number runs from 5.25e+4 at "n6" to 1.72e+5 at "n12", a factor of 3.28, which is 0.52 decimal digits of the arithmetic's own margin.

The node that is not in the circuit

Nodal analysis needs a node to call zero and no circuit contains one. Moving it changes every node voltage by the same amount and no branch voltage or branch current at all — to a part in ten to the fourteenth on a well-behaved chain. What it does change is the matrix: the condition number of a twelve-section chain moves by a factor of 3.3 with the reference, and on a star whose resistances span nine decades by 8.0. Measured against the same matrix solved in twice the precision, the worst reference costs four decimal digits of the answer, and it is the reference the condition number named before the error was looked at.

networks · Nodal analysis
Stepped at 20 of its time constant, a 1 µs pole rings between 1.818 and 0.331 V, and needs 23 steps to settle. Marched with the trapezoidal rule at a step of 20.0 µs. A 1 µs pole (1 kΩ, 1 nF) drives, through a unity buffer, a 1 ms pole (1 kΩ, 1 µF). The fast node's exact response reaches its final volt within a few microseconds; the march's first values are 1.8182, 0.3306, 1.5477, 0.5519, 1.3666 V. Its distance from its final volt is multiplied by (1 − h/2τ)/(1 + h/2τ) = −0.8182 every step, measured and checked against that form, so it changes sign every step and takes 23 steps to fall below 1% — 460 µs. The slow node it drives is 1.23e-5 V from exact at 1 ms, because a 1 ms pole averages an alternation at half the stepping rate to nothing.

The ringing that belongs to the rule

The trapezoidal rule is stable for every stable circuit and every step size, and it is not damping. March a one-microsecond pole with twenty-microsecond steps and its node reads 1.818, 0.331, 1.548, 0.552 volts — an oscillation at half the stepping rate, its distance from the final volt multiplied by exactly −0.8182 every step, taking twenty-three steps to fall below one per cent. The slow node that pole drives is right to 1.2 × 10⁻⁵ V at a millisecond. One backward-Euler step at the discontinuity cuts the first swing from 0.818 V to 0.048 and two to 0.0023, because backward Euler multiplies the same error by 1/(1 + h/τ) and the trapezoidal rule by (1 − h/2τ)/(1 + h/2τ), which approaches −1.

transients · Step response
At 20 steps a cycle, ten cycles of an undamped LC: the trapezoidal rule keeps the amplitude and falls 29.2° behind; backward Euler keeps 0.0082% of it. Marched, both rules, against 1 − cos ωt for a 1 kHz inductor–capacitor pair stepped with no resistance at all. At 20 steps a cycle the trapezoidal march's amplitude stays at 1.00000 a cycle and its frequency is slow: it loses 2.918° a cycle, measured from the march's own recurrence, against 2π − 2N·atan(π/N) = 2.918°, so after ten cycles it is 29.2° behind. Backward Euler keeps 0.3901 of its amplitude a cycle, against (1 + (2π/N)²)^(−N/2) = 0.3901, so 0.0082% is left after ten, and it loses 11.19° a cycle. No resistance is in the circuit; every loss is the rule's.

The phase the rule loses

An inductor and a capacitor with no resistance ring for ever, and two ways of marching them disagree about how. The trapezoidal rule keeps the amplitude exactly — its factor per step has a magnitude of one — and loses phase instead: 2π − 2N·atan(π/N) a cycle, 2.918° at twenty steps a cycle, so ten cycles later it is 29.2° behind the circuit. Backward Euler keeps 0.3901 of the amplitude a cycle at the same step, and after ten cycles 0.0082 per cent of the ringing is left, in a circuit that has no loss. The two errors fall at different rates: the trapezoidal rule's phase as the square of the steps a cycle, backward Euler's amplitude as the first power. A hundred cycles to within one per cent needs 182 steps a cycle of one and 196,404 of the other.

transients · Step response
The best foil thickness for 4 layers, for three currents with the same fundamental. computed by solving, not by drawing. The loss of a portion of 4 layers against foil thickness, with the loss weighted by the current in each harmonic rather than computed for one frequency. A sinusoid wants 0.6631 skin depths and lands at 1.3368 times the direct-current resistance — four thirds, the rung below's constant, reproduced. A triangular ripple wants 0.6432, which is the same answer to within 3.0 per cent, so a winding carrying one needs none of this. A square current of the same fundamental wants 0.3838 — thinner by a factor of 1.728 — and lands at 1.8313, which is not four thirds and is not any constant the geometry knows. Building to the sinusoid's answer costs 16.8 per cent more loss.

The optimum a spectrum moves

The best foil thickness for a winding is derived for one sinusoid and quoted as a property of the geometry: a minimum at four thirds of the direct-current resistance, whatever the layer count. Weight the loss by the current in each harmonic instead and a square current of the same fundamental wants foil 1.728 times thinner and lands at 1.83, and a narrow pulse wants it 3.68 times thinner. Four thirds is a property of the current. The constant that replaces it for an ideal square edge is exactly two, and a real winding sits between them at a place its edge rate decides.

magnetics · Winding

Named alongside it

The objects these essays reach for when they reach for this one.

VerificationModel rangeConvergence orderModel refusalRealisationModified nodal analysisSingular matrixClosed formComponent toleranceConditioningDesign tradeoffResidues

All concepts