Resources Numerical accuracy Calculation policy

Library configurator sizes

The size and connection positions shown for the chosen variant on a Library item's configure page.

How the precision is kept

The numbers come from the server's item data as 64-bit numbers. The page only turns them into text, rounded to 4 decimal places of a millimeter. The same rounding is used to match a selected option to a variant.

Calculations under this policy

2 calculations in 1 source file. The code is read from the released app when this page loads. It is the calculation people run, not a copy kept for this page.

library-configurator.js Shared shell, every app 64-bit 2 calculations Reviewed

Tests None found by the review

Size and position text

64-bit

A dimension, port size or port position as text in millimeters.

text = round(x × 10⁴) ÷ 10⁴

Rounds the text only; the variant data is not changed. The product x × 1e4 is itself rounded to 64-bit before Math.round, so a value within one rounding step of a tie can go either way.

  • library-configurator.js
  • line 38
const fmtNum = (x) => String(Math.round(x * 1e4) / 1e4);

Option value used for matching

64-bit

The text an option value is compared by when the page picks the variant that matches the selected options.

numbers: round(v × 10⁴) ÷ 10⁴ as text; other values: trimmed text

Numeric option values are compared after rounding to 4 decimals, as text. Values that round to the same 4 decimals count as the same option.

  • library-configurator.js
  • line 20
const text = (v) => (typeof v === 'number' ? String(Math.round(v * 1e4) / 1e4) : String(v ?? '').trim());

In detail

Method
Round to 4 decimals: Math.round(x × 10⁴) ÷ 10⁴, then String().
Number types
64-bit numbers.
Limits
Shown values have at most 4 decimal places. Two numeric option values that round to the same 4 decimals compare as equal when a variant is chosen.

Approach: 64-bit, direct

Every value is a 64-bit number from input to result. Formulas are closed form or a direct solve, and every comparison uses a stated tolerance.

Every policy that takes this approach