Resources Numerical accuracy

Calculations

Every calculation under our calculation policies: what it works out, its formula and the precision it runs at. Open one to see its code as it is in the released app.

  • 6,917 calculations in 1,248 source files
  • 143 exact
  • 6,286 in 64-bit
  • 335 with a 32-bit step
  • Last reviewed

Calculations

  • Pin header

    Drilled pads in one or two rows at a pitch.

    x = (r − (rows − 1) ÷ 2) × pitch; y = ((n − 1) ÷ 2 − i) × pitch

    Parametric PCB footprints footlib.js Every app
    64-bit
  • Quad flat package

    Pads on four sides, numbered counter-clockwise from the top left.

    along a side: ((n − 1) ÷ 2 − i) × pitch; across: ± span ÷ 2

    Parametric PCB footprints footlib.js Every app
    64-bit
  • Axial through-hole part

    Two drilled pads at half the lead pitch either side of the center.

    x = ∓ pitch ÷ 2

    Parametric PCB footprints footlib.js Every app
    64-bit
  • Positions stored as offsets from the origin

    The kept points as 32-bit offsets from a whole-unit origin.

    pos = fround(x − floor(min))

    Point cloud positions and read-in values pointcloud.js
    • CadCraft
    • BimCraft
    32-bit in part
  • A point back in file coordinates

    The file-unit position of one kept point.

    p = pos + origin

    Point cloud positions and read-in values pointcloud.js
    • CadCraft
    • BimCraft
    32-bit in part
  • Points in a slice

    The points that lie within a thickness of a plane.

    |pos − (at − origin)| ≤ thick ÷ 2

    Point cloud positions and read-in values pointcloud.js
    • CadCraft
    • BimCraft
    32-bit in part
  • Intensities scaled to 0 to 65535

    The intensity of each kept point on a common 16-bit scale.

    out = round((v − lo) ÷ (hi − lo) × 65535)

    Point cloud positions and read-in values pointcloud.js
    • CadCraft
    • BimCraft
    32-bit in part
  • LAS record decoder

    The x, y, z of each LAS point from its stored integers.

    x = Int32 × scale + offset

    Point cloud positions and read-in values pointcloud.js
    • CadCraft
    • BimCraft
    64-bit
  • Size of the cloud

    The extent of the cloud in each axis shown in the summary.

    size = max − min

    Point cloud positions and read-in values pointcloud.js
    • CadCraft
    • BimCraft
    64-bit
  • Ray against boxes

    Calls back for every item whose box a ray passes through.

    t₀ = max(nearₐ), t₁ = min(farₐ); hit if t₀ ≤ t₁

    Box tree for picking and collisions m3d/bvh.js
    • CadCraft
    • ModelCraft
    • PartCraft
    64-bit
  • Boxes of point lists

    The box of each item from its points, grown by a pad.

    Box tree for picking and collisions m3d/bvh.js
    • CadCraft
    • ModelCraft
    • PartCraft
    64-bit
  • Summed K of a list of fittings

    The total loss coefficient of the fittings on one link at its own Reynolds number and diameter.

    K = Σ nᵢ × Kᵢ(Re, D)

    Fitting losses summed from a pack process/equip/fittingk.js
    • CadCraft
    • BimCraft
    64-bit
  • Head loss of a fitting

    The head loss in meters from K and the velocity.

    h = K × v² ÷ (2g)

    Fitting losses summed from a pack process/equip/fittingk.js
    • CadCraft
    • BimCraft
    64-bit
  • Pressure thrust

    The axial force on a blind end or an unbalanced joint.

    F = P × π × Di² ÷ 4

    Pipe stress support helpers process/equip/pipestress2.js
    • CadCraft
    • BimCraft
    64-bit
  • Variable spring selection

    The spring sizes that carry the hot load and travel within the allowed load variation.

    variation = |travel| × rate ÷ hot load

    Pipe stress support helpers process/equip/pipestress2.js
    • CadCraft
    • BimCraft
    64-bit
  • Stop clearance check

    Whether the thermal displacement closes a gap or stop.

    Pipe stress support helpers process/equip/pipestress2.js
    • CadCraft
    • BimCraft
    64-bit
  • Length of a sew-line edge

    The length of one edge of the sew line at a size.

    Pattern piece lengths, seam walking and areas addons/apparel/lib/measure.js
    • CadCraft
    64-bit
  • Distance round the outline between two points

    The forward and backward length along the sew line between two snapped points.

    fwd = ((d_b − d_a) mod L + L) mod L; back = (L − fwd) mod L

    Pattern piece lengths, seam walking and areas addons/apparel/lib/measure.js
    • CadCraft
    64-bit
  • Seam walking

    The lengths of two edges, their difference and the notch offsets along the seam.

    diff = lenB − lenA; off = bₖ − aₖ

    Pattern piece lengths, seam walking and areas addons/apparel/lib/measure.js
    • CadCraft
    64-bit
  • Perimeter and area per size

    The sew and cut perimeter and area of each size of a piece.

    area = |Σ shoelace|

    Pattern piece lengths, seam walking and areas addons/apparel/lib/measure.js
    • CadCraft
    64-bit
  • Meets the limit

    True, false or null for a found value against a rule.

    min − e ≤ found ≤ max + e

    Pass or fail against a regulation limit addons/regs/lib/rows.js
    • CadCraft
    • BimCraft
    • CivilCraft
    • DetailCraft
    64-bit
  • Material and band thicknesses

    The thickness of each library sheet and edge band in millimeters and in inches.

    Cabinet material sizes, grooves and drill files addons/cabinetcraft/lib/materials.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • PartCraft
    Exact
  • Rules from the material picks

    The thicknesses, groove and band thicknesses that the construction rules use.

    groove = min(tback, 0.6 × tcarcass)

    Cabinet material sizes, grooves and drill files addons/cabinetcraft/lib/materials.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • PartCraft
    64-bit
  • Sheet size for a material

    The sheet width and height a material is cut from.

    in = round(mm ÷ 25.4 × 4) ÷ 4

    Cabinet material sizes, grooves and drill files addons/cabinetcraft/lib/materials.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • PartCraft
    64-bit
  • Hole positions in the drill XML

    The x and y of each hole as written in the drill file.

    x = round(x × 1000) ÷ 1000

    Cabinet material sizes, grooves and drill files addons/cabinetcraft/lib/export.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • PartCraft
    64-bit
  • G-code scale and thickness

    The unit scale and cut depth passed to the sheet G-code writer.

    thickness_mm = thickness × scale

    Cabinet material sizes, grooves and drill files addons/cabinetcraft/lib/export.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • PartCraft
    64-bit
  • Relative difference

    The fraction by which a coarse result differs from the refined result.

    |a − b| ÷ |b|

    Spread between coarse and refined results study/live.js
    • CadCraft
    • BimCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Vertices and loose edges

    The point positions of a model definition and its loose edges.

    Model points written to USD usd/m3d.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    32-bit in part
  • Mesh arrays

    The point, normal, UV and color arrays of a mesh.

    Model points written to USD usd/m3d.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    32-bit in part
  • Number to text

    The text a number is written as, by type.

    f32: text = shortest s with fround(+s) = fround(x)

    Number text in USD files usd/usda-write.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    32-bit in part
  • Shortest digits

    A number to p significant digits without trailing zeros.

    Number text in USD files usd/usda-write.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Type name to number type

    Which number type a USD type name uses (32-bit float, half or 64-bit double).

    Number text in USD files usd/sdf.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    Exact
  • Inline values

    A value that the file stores inside its 32-bit payload word.

    Floats read from USD files usd/usdc-read.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    32-bit in part
  • Raw arrays

    A packed array of numbers of one type.

    Floats read from USD files usd/usdc-read.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    32-bit in part
  • Compressed float arrays

    An array stored as whole numbers or as a lookup table and indexes.

    Floats read from USD files usd/usdc-read.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    32-bit in part
  • Float text rounding

    A float or half value read from .usda text.

    Floats read from USD files usd/usda-read.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    32-bit in part
  • Arrays from text

    A list of parsed numbers as a typed array of the right type.

    Floats read from USD files usd/usda-read.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    32-bit in part
  • Half to float

    The value of a 16-bit half float.

    v = ± (1 + m ÷ 1024) × 2^(e − 15)

    Half floats in USD usd/util.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    32-bit in part
  • Float to half

    The 16-bit half closest to a number.

    Half floats in USD usd/util.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    32-bit in part
  • Measure to SI

    A property value in meters, square meters or cubic meters.

    area = v × L²; volume = v × L³

    IFC measures in SI for USD usd/ifc.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Significand width of a number

    How many bits of a number this device keeps.

    bits = the first n where 1 + 2⁻ⁿ = 1

    Device number check accuracy.js Every app
    Exact
  • Digits of a graphics float

    The number of significant digits stated for the graphics processor's float type.

    bits < 23: digits = max(1, ⌊(bits + 1) × log₁₀ 2⌋); bits ≥ 23: the fixed text "about 7"

    Device number check accuracy.js Every app
    64-bit
  • Size and position text

    A dimension, port size or port position as text in millimeters.

    text = round(x × 10⁴) ÷ 10⁴

    Library configurator sizes library-configurator.js Every app
    64-bit
  • Option value used for matching

    The text an option value is compared by when the page picks the variant that matches the selected options.

    numbers: round(v × 10⁴) ÷ 10⁴ as text; other values: trimmed text

    Library configurator sizes library-configurator.js Every app
    64-bit
  • Mesh buffers

    The item's box or mesh as geometry for drawing, turned from z up to y up.

    Library 3D preview library-3d.js Every app
    32-bit, display only
  • Camera position

    Where the camera sits as the view is turned and zoomed.

    position = target + dist × (sin φ sin θ, cos φ, sin φ cos θ)

    Library 3D preview library-3d.js Every app
    32-bit, display only
  • Variant body buffers

    The chosen variant's box or mesh as geometry for drawing.

    Library 3D preview library-configurator.js Every app
    32-bit, display only
  • Required and found values as text

    A limit or a measured value in words, such as ≤ 196.85 mm or 863.6–965.2 mm.

    text = round(x × 100) ÷ 100, then the unit

    Check report values problems.js Every app
    64-bit
  • Built-in unit factors

    The factor from each built-in unit of mass, time, force, pressure, energy, power, current, voltage and volume to SI.

    lb = 0.45359237 kg; lbf = 4.4482216152605 N; psi = 6894.757293168 Pa; psf = 47.88025898 Pa

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Constants and fallback lengths

    pi, e, standard gravity g0, and the length units used when no unit table is given.

    g0 = 9.80665 m/s²; in = 0.0254 m; ft = 0.3048 m; yd = 0.9144 m

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit

A calculation is added only after its source was read. Each statement can be checked in the code shown on its policy's page; the app repositories are public. When a release changes a calculation, its page says so until it has been read again.