Resources Numerical accuracy

Calculations

Every calculation under our calculation policies: what it works out, its formula and the precision it runs at. Open one to see its code as it is in the released app.

  • 6,917 calculations in 1,248 source files
  • 143 exact
  • 6,286 in 64-bit
  • 335 with a 32-bit step
  • Last reviewed

Calculations

  • Weld grid for splitting shells

    Merges corners at the same place so separate pieces of a mesh can be told apart.

    q = max(range, 1e-9) × 1e-7; key = round(x ÷ q), round(y ÷ q), round(z ÷ q)

    Manual: imported product meshes, copies and part volume illustrate/import.js
    • PartCraft
    64-bit
  • Part signature

    A short key (corner count, triangle count, area, volume) that groups parts that may be copies.

    Manual: imported product meshes, copies and part volume illustrate/import.js
    • PartCraft
    32-bit in part
  • Recognizing copies of a part

    Folds parts that are rigid moves of one another into one part with several placements.

    copy when fit error ≤ 1e-5 × size and largest corner difference ≤ 4e-5 × size

    Manual: imported product meshes, copies and part volume illustrate/import.js
    • PartCraft
    64-bit
  • Largest corner difference after a move

    How far the moved corners of one mesh are from another's.

    Manual: imported product meshes, copies and part volume illustrate/import.js
    • PartCraft
    64-bit
  • Placement moved to millimeters

    A 3MF placement with its move in millimeters.

    Manual: imported product meshes, copies and part volume illustrate/import.js
    • PartCraft
    64-bit
  • Mesh in the manual file

    A part's mesh as written into the manual file.

    Manual: imported product meshes, copies and part volume illustrate/manual.js
    • PartCraft
    32-bit in part
  • Explode offsets

    How far each part has moved at a point in an exploded view.

    offset = Σ dir × dist × clamp(t × n − k, 0, 1) over the steps k

    Manual: imported product meshes, copies and part volume illustrate/manual.js
    • PartCraft
    64-bit
  • Where each part is drawn

    Each instance's matrix in a view.

    M = translation(explode offset + view move) × placement

    Manual: imported product meshes, copies and part volume illustrate/manual.js
    • PartCraft
    64-bit
  • Moving parts in a view

    Adds a move to parts in one view.

    Manual: imported product meshes, copies and part volume illustrate/manual.js
    • PartCraft
    64-bit
  • Part volume and area

    The volume and surface area of a part's mesh.

    V = |Σ a·(b × c) ÷ 6|; A = Σ |(b − a) × (c − a)| ÷ 2

    Manual: imported product meshes, copies and part volume illustrate/manual.js
    • PartCraft
    32-bit in part
  • Projection onto the picture

    A model point's place in the picture and its depth.

    orthographic: u = aspect ÷ 2 + q·right ÷ 2h; perspective: u = aspect ÷ 2 + q·right ÷ (2 tan(fov ÷ 2) × depth)

    Illustrate: projection, hidden lines and vector pictures illustrate/camera.js
    • PartCraft
    32-bit, display only
  • Is a picture point hidden

    Decides whether a point of an edge is behind a triangle of another surface.

    barycentric l₁, l₂, l₃ in the picture; hidden when the triangle's depth < point depth − 10 eps

    Illustrate: projection, hidden lines and vector pictures illustrate/hlr.js
    • PartCraft
    32-bit, display only
  • Feature edges

    The mesh edges drawn as lines: open edges and edges where two faces meet at more than an angle.

    drawn when open, or cos(face angle) < cos(angle)

    Illustrate: projection, hidden lines and vector pictures illustrate/edges.js
    • PartCraft
    32-bit, display only
  • Numbers in the PDF

    How a coordinate is written into a PDF page.

    Illustrate: projection, hidden lines and vector pictures illustrate/pdf.js
    • PartCraft
    32-bit, display only
  • Significand width of a number

    How many bits of a number this device keeps.

    bits = the first n where 1 + 2⁻ⁿ = 1

    Device number check accuracy.js Every app
    Exact
  • Digits of a graphics float

    The number of significant digits stated for the graphics processor's float type.

    bits < 23: digits = max(1, ⌊(bits + 1) × log₁₀ 2⌋); bits ≥ 23: the fixed text "about 7"

    Device number check accuracy.js Every app
    64-bit
  • Size and position text

    A dimension, port size or port position as text in millimeters.

    text = round(x × 10⁴) ÷ 10⁴

    Library configurator sizes library-configurator.js Every app
    64-bit
  • Option value used for matching

    The text an option value is compared by when the page picks the variant that matches the selected options.

    numbers: round(v × 10⁴) ÷ 10⁴ as text; other values: trimmed text

    Library configurator sizes library-configurator.js Every app
    64-bit
  • Mesh buffers

    The item's box or mesh as geometry for drawing, turned from z up to y up.

    Library 3D preview library-3d.js Every app
    32-bit, display only
  • Camera position

    Where the camera sits as the view is turned and zoomed.

    position = target + dist × (sin φ sin θ, cos φ, sin φ cos θ)

    Library 3D preview library-3d.js Every app
    32-bit, display only
  • Variant body buffers

    The chosen variant's box or mesh as geometry for drawing.

    Library 3D preview library-configurator.js Every app
    32-bit, display only
  • Required and found values as text

    A limit or a measured value in words, such as ≤ 196.85 mm or 863.6–965.2 mm.

    text = round(x × 100) ÷ 100, then the unit

    Check report values problems.js Every app
    64-bit
  • Built-in unit factors

    The factor from each built-in unit of mass, time, force, pressure, energy, power, current, voltage and volume to SI.

    lb = 0.45359237 kg; lbf = 4.4482216152605 N; psi = 6894.757293168 Pa; psf = 47.88025898 Pa

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Constants and fallback lengths

    pi, e, standard gravity g0, and the length units used when no unit table is given.

    g0 = 9.80665 m/s²; in = 0.0254 m; ft = 0.3048 m; yd = 0.9144 m

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Unit lookup

    The SI factor and dimension of a unit name.

    deg = π ÷ 180; rad = 1

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Dimension comparison

    Whether two values have the same dimensions, so they can be added, compared or converted.

    |aᵢ − bᵢ| < 1e-9 for each of the five exponents

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Power

    a ^ b for a value with units.

    value = aᵇ; dimensions × b

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Conversion with to

    M to kN m: checks the dimension and records the unit to show the answer in.

    shown = value ÷ factor of the target unit

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Shown amount

    The number and unit a row displays, and the amount passed to other modules.

    amount = value ÷ factor

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Square root and absolute value

    sqrt(a) and abs(a) for a value with units.

    √a halves the dimensions

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Rounding to a step

    round, floor and ceil, with an optional step such as round(x, 5 mm).

    round(a ÷ s) × s

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Minimum, maximum, sum and average

    min, max, sum and avg of values of one kind.

    sum = Σ aᵢ in order; avg = sum ÷ n

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Trigonometry, logarithms and exponent

    sin, cos, tan, their inverses, atan2, ln, log10, exp and pow.

    sin, cos, tan of radians; asin, acos need |x| ≤ 1; ln, log10 need x > 0

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Reading an app parameter

    @Name: the app's parameter as a value in SI.

    value = parameter value × factor of its unit

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Values written to driven parameters

    What each drive would set in the app: the row's answer in the parameter's unit.

    value = row value ÷ factor of the unit; change when |current − value| > 1e-9 × max(1, |value|)

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Time-weighted average

    The average of a value that changes at events: queue length, units busy, work in process.

    average = (Σ vᵢ × Δtᵢ) ÷ (t − t₀)

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Mean, deviation and percentiles of observations

    Waiting time, service time and time in system: mean, standard deviation, minimum, maximum, P50 and P95.

    m ← m + (x − m) ÷ n; s ← s + (x − m_old)(x − m_new); sd = √(s ÷ (n − 1)); P = sorted[⌊p × k⌋]

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Event time

    When a scheduled event happens.

    t_event = t_now + delay

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Conveyor slots and timing

    How many items fit on a belt, the gap between items and the travel time.

    slots = max(1, ⌊length ÷ pitch + 1e-9⌋); headway = pitch ÷ speed; travel = length ÷ speed

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Shift edges

    When a server goes on and off shift.

    x = t − ⌊t ÷ period⌋ × period; on shift when a ≤ x < b

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Server utilization

    The share of time a server's units were busy, blocked, down and idle.

    utilization = average busy units ÷ units; idle = max(0, 1 − busy − blocked − down)

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Throughput

    Finished entities per unit of time at a sink.

    throughput = count ÷ (t − t₀)

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Resource utilization

    The share of a shared resource's capacity that was in use.

    utilization = average in use ÷ average capacity

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Vehicle trip times

    How long a vehicle takes to reach the pick station and to carry the load to the drop station.

    to pick = distance ÷ speed + load time; carry = distance ÷ speed + unload time

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Student t factor

    The factor for a 95 % confidence range.

    df ≤ 30: table value; df ≤ 60: 2.0; df ≤ 120: 1.98; above: 1.96

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Replication mean and confidence half-width

    The mean of each result over independent runs and its 95 % range.

    mean = Σx ÷ n; sd = √(Σ(x − mean)² ÷ (n − 1)); half = t × sd ÷ √n

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Uniform and normal draws

    The random numbers every arrival, service time and route choice is drawn from.

    u = (k + 0.5) ÷ 2³², k a 32-bit integer; z = √(−2 ln u) × cos(2π v)

    Flow simulation (discrete events) rng.js Every app
    64-bit
  • Draw from a distribution

    One value from an exponential, uniform, triangular, normal, lognormal, Erlang, Weibull or empirical distribution.

    exp: −mean × ln u; tri: a + √(u (b − a)(c − a)) or b − √((1 − u)(b − a)(b − c)); weibull: scale × (−ln u)^(1 ÷ shape)

    Flow simulation (discrete events) rng.js Every app
    64-bit
  • Mean of a distribution

    The mean used for load estimates and for routing to the least loaded target.

    tri: (a + m + b) ÷ 3; weibull: scale × Γ(1 + 1 ÷ shape)

    Flow simulation (discrete events) rng.js Every app
    64-bit
  • Gamma function

    Γ(x), used for the mean of a Weibull distribution.

    Lanczos approximation, g = 7, 9 coefficients

    Flow simulation (discrete events) rng.js Every app
    64-bit

A calculation is added only after its source was read. Each statement can be checked in the code shown on its policy's page; the app repositories are public. When a release changes a calculation, its page says so until it has been read again.