Two independent implementations produce identical matched-operation FFT outputs at 113 and 256 significant bits. Native MATLAB and Advanpix FFTs are also checked against exact integer references in 48 overlap-add and overlap-save cases.
Test environment: MATLAB R2020b Update 2 and Advanpix 5.1.0.15432, with eight configured Advanpix threads.
Precision and reference checks.
- mp.Digits(34), mp.GuardDigits(0): 113 significant binary bits.
- mp.Digits(77), mp.GuardDigits(0): 256 significant binary bits.
- Precision verified by spacing at 1 and halfway rounding checks.
- Exact signed64 operands and convolution references imported from binary integer files; the value 2^53+1 explicitly verified to survive import exactly.
- Outputs and exact references both promoted to 200 decimal digits (665 bits) BEFORE subtraction. No residual is formed in double precision.
- The native tests use single-number floating-point precision. The double-double illustration uses a separate two-double representation.
Two different checks.
- Matched arithmetic: a 512-tap filter, 1,024-point FFT and two input blocks are evaluated in MATLAB/Advanpix and Python/mpmath. Both use identical rounded phase factors and the same sequence of arithmetic operations. All 1,537 output samples agree exactly at 113 and 256 significant bits.
- Native FFTs: MATLAB double and Advanpix at 113 and 256 significant bits are tested at 512, 2,048, 8,192 and 16,384 filter taps. White-noise and transient signals use both overlap-add and overlap-save. Every output sample in all 48 test cases is compared with exact integer references.
Measured native peak absolute errors at 16k taps / 32k FFT.
| Arithmetic | White OLA | White OLS | Transients OLA | Transients OLS |
|---|---|---|---|---|
| MATLAB IEEE double | 2.776e-16 | 3.331e-16 | 5.551e-17 | 5.551e-17 |
| Advanpix 113-bit significand | 2.889e-34 | 2.889e-34 | 4.815e-35 | 9.630e-35 |
| Advanpix 256-bit significand | 1.916e-77 | 2.159e-77 | 4.148e-78 | 4.318e-78 |
Native versus custom arithmetic.
At a fixed working precision, a different FFT algorithm can produce different rounding errors. A faster or more mature library is not automatically more accurate for every input.
| Significand bits | Native minus custom RMS error, measured dB range |
|---|---|
| 53 | -1.44 to +0.67 |
| 113 | +0.77 to +2.49 |
| 256 | -1.78 to -0.16 |
Positive means native error is larger; negative means smaller. All comparisons use identical stored operands and exact references.
At 16K taps, 113-bit filtering takes 0.049–0.069 seconds per case; 256-bit filtering takes 0.213–0.326 seconds. These single-run timings include the filter FFT and overlap processing, but exclude startup, reference generation and error analysis.
Reference checks.
Stored inputs and filter coefficients are identical across the compared implementations. Exact integer convolution provides the reference, and output differences are calculated at 665-bit precision. The checks cover precision settings, exact input conversion, arithmetic agreement and measured native filtering error.
Reference: Advanpix precision controls and FFT function reference.