Resources Numerical accuracy

Calculations

Every calculation under our calculation policies: what it works out, its formula and the precision it runs at. Open one to see its code as it is in the released app.

  • 6,917 calculations in 1,248 source files
  • 143 exact
  • 6,286 in 64-bit
  • 335 with a 32-bit step
  • Last reviewed

Calculations

  • Deepest water

    The deepest water anywhere in the run, as shown in the facts list.

    deepest = max over cells of maxDepth, converted to mm or in

    Rainfall run read-outs: deepest water and depth scale rain-ui.js
    • CivilCraft
    32-bit in part
  • Depth scale of the picture

    The depth at which the picture reaches its deepest color, printed in the legend.

    top = max(0.02, 99.5th percentile of wet maxDepth cells)

    Rainfall run read-outs: deepest water and depth scale rain-ui.js
    • CivilCraft
    32-bit in part
  • Assistant rounding to 3 decimals

    Rounds a result to 3 decimals before the assistant is given it.

    round(v × 1000) ÷ 1000

    Assistant and window read-outs of radio link and coverage geo-agent.js
    • CivilCraft
    64-bit
  • Coverage share and map

    The share of the area covered, the farthest covered point and a coarse map of received power.

    map cell = r3(p) when p ≥ sensitivity, else null

    Assistant and window read-outs of radio link and coverage geo-agent.js
    • CivilCraft
    32-bit in part
  • Coverage read-out in the window

    The share covered, the farthest covered distance and the cell count shown for a coverage item.

    covered % = covered × 100; farthest = distance as returned

    Assistant and window read-outs of radio link and coverage geo-ui.js
    • CivilCraft
    32-bit in part
  • Array position from the coordinate system

    The latitude and longitude of the middle of the array's region.

    c = mean of corners; (lon, lat) = toLonLat(c); rounded to 1e-5°

    Solar array site and saved summary solar-run.js
    • CivilCraft
    64-bit
  • Saved plan summary

    The counts, kWp and yearly kWh the file keeps for an array.

    kWh = round(annual AC kWh)

    Solar array site and saved summary solar-run.js
    • CivilCraft
    64-bit
  • Area read-out

    Prints an area in the project's area unit.

    text = round to 0 decimals, grouped

    Massing window read-outs and typed height massing-ui.js
    • CivilCraft
    64-bit
  • Typed height to floors

    Turns a height the person types into a whole number of floors.

    floors = clamp(round(H ÷ floor height), 1, max floors)

    Massing window read-outs and typed height massing-ui.js
    • CivilCraft
    64-bit
  • Zoning coverage percent

    Turns the percent a person types for building coverage into the stored fraction.

    coverage = n ÷ 100

    Massing window read-outs and typed height massing-ui.js
    • CivilCraft
    64-bit
  • Bench rim line segments

    Segment pairs for the rim of each bench.

    Pit and solar table meshes in the 3D view site3d-pit.js
    • CivilCraft
    32-bit, display only
  • Haul ramp ribbon

    A road-wide ribbon on the pit walls for the haul ramp.

    offset = (−dy, dx) ÷ length × width ÷ 2; lift = 0.02 × width

    Pit and solar table meshes in the 3D view site3d-pit.js
    • CivilCraft
    32-bit, display only
  • Solar table triangle normal

    A flat-shaded triangle with its normal.

    n = (b − a) × (c − a) ÷ |…|

    Pit and solar table meshes in the 3D view site3d-solar.js
    • CivilCraft
    32-bit, display only
  • Pit scene in project units

    Converts a pit's ramp and bench outlines from meters back to project units for the 3D view.

    coordinate ÷ k, where k is meters per project unit

    Pit and solar table meshes in the 3D view pit-view.js
    • CivilCraft
    32-bit, display only
  • Finding the sheet

    The four corners of the paper in the photo.

    corners = the pixels of the largest bright region with the smallest and largest x + y and x − y; photo position = (pixel + 0.5) × reduction factor

    Sketch photo to drawing: sheet, perspective and scale photosketch/photosketch.js
    • ModelCraft
    • CivilCraft
    64-bit
  • Perspective transform from four points

    The 3 × 3 matrix that takes four points to four others.

    8 equations in h₁…h₈ with h₉ = 1, solved by Gauss–Jordan elimination

    Sketch photo to drawing: sheet, perspective and scale photosketch/photosketch.js
    • ModelCraft
    • CivilCraft
    64-bit
  • Applying the perspective transform

    A point taken through the 3 × 3 matrix.

    w = h₆x + h₇y + h₈; x′ = (h₀x + h₁y + h₂) ÷ w; y′ = (h₃x + h₄y + h₅) ÷ w

    Sketch photo to drawing: sheet, perspective and scale photosketch/photosketch.js
    • ModelCraft
    • CivilCraft
    64-bit
  • Sheet proportions

    The long-to-short ratio of the straightened sheet.

    w = mean of top and bottom lengths, h = mean of left and right; r = max ÷ min; replaced by the named paper's ratio, or by the nearest standard ratio within 8 %

    Sketch photo to drawing: sheet, perspective and scale photosketch/photosketch.js
    • ModelCraft
    • CivilCraft
    64-bit
  • Drawing units per pixel

    The size of one pixel of the straightened sheet in drawing units, and the mapping from pixels to drawing coordinates.

    mm per px = long side in mm ÷ long side in px; units per px = mm per px × scale ÷ unit in mm; x = origin_x + px·k, y = origin_y + (height − py)·k

    Sketch photo to drawing: sheet, perspective and scale photosketch/photosketch.js
    • ModelCraft
    • CivilCraft
    64-bit
  • Arcs and circles into the drawing

    Converts each traced arc to drawing units as a circle or as an arc through its start, middle and end.

    r = r_px × units per px; circle when a₁ − a₀ ≥ 2π − 1e-6; else arc through start, middle (at (a₀ + a₁) ÷ 2) and end

    Sketch photo to drawing: sheet, perspective and scale photosketch/photosketch.js
    • ModelCraft
    • CivilCraft
    64-bit
  • Arc through three points

    The center, radius and end angles of the arc through three points.

    d = 2·(a_x(b_y − c_y) + b_x(c_y − a_y) + c_x(a_y − b_y)); center from the sums of |p|²·(differences) ÷ d; r = |a − center|

    Sketch photo to drawing: sheet, perspective and scale photosketch/paper.js
    • ModelCraft
    • CivilCraft
    64-bit
  • Significand width of a number

    How many bits of a number this device keeps.

    bits = the first n where 1 + 2⁻ⁿ = 1

    Device number check accuracy.js Every app
    Exact
  • Digits of a graphics float

    The number of significant digits stated for the graphics processor's float type.

    bits < 23: digits = max(1, ⌊(bits + 1) × log₁₀ 2⌋); bits ≥ 23: the fixed text "about 7"

    Device number check accuracy.js Every app
    64-bit
  • Size and position text

    A dimension, port size or port position as text in millimeters.

    text = round(x × 10⁴) ÷ 10⁴

    Library configurator sizes library-configurator.js Every app
    64-bit
  • Option value used for matching

    The text an option value is compared by when the page picks the variant that matches the selected options.

    numbers: round(v × 10⁴) ÷ 10⁴ as text; other values: trimmed text

    Library configurator sizes library-configurator.js Every app
    64-bit
  • Mesh buffers

    The item's box or mesh as geometry for drawing, turned from z up to y up.

    Library 3D preview library-3d.js Every app
    32-bit, display only
  • Camera position

    Where the camera sits as the view is turned and zoomed.

    position = target + dist × (sin φ sin θ, cos φ, sin φ cos θ)

    Library 3D preview library-3d.js Every app
    32-bit, display only
  • Variant body buffers

    The chosen variant's box or mesh as geometry for drawing.

    Library 3D preview library-configurator.js Every app
    32-bit, display only
  • Required and found values as text

    A limit or a measured value in words, such as ≤ 196.85 mm or 863.6–965.2 mm.

    text = round(x × 100) ÷ 100, then the unit

    Check report values problems.js Every app
    64-bit
  • Built-in unit factors

    The factor from each built-in unit of mass, time, force, pressure, energy, power, current, voltage and volume to SI.

    lb = 0.45359237 kg; lbf = 4.4482216152605 N; psi = 6894.757293168 Pa; psf = 47.88025898 Pa

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Constants and fallback lengths

    pi, e, standard gravity g0, and the length units used when no unit table is given.

    g0 = 9.80665 m/s²; in = 0.0254 m; ft = 0.3048 m; yd = 0.9144 m

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Unit lookup

    The SI factor and dimension of a unit name.

    deg = π ÷ 180; rad = 1

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Dimension comparison

    Whether two values have the same dimensions, so they can be added, compared or converted.

    |aᵢ − bᵢ| < 1e-9 for each of the five exponents

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Power

    a ^ b for a value with units.

    value = aᵇ; dimensions × b

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Conversion with to

    M to kN m: checks the dimension and records the unit to show the answer in.

    shown = value ÷ factor of the target unit

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Shown amount

    The number and unit a row displays, and the amount passed to other modules.

    amount = value ÷ factor

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Square root and absolute value

    sqrt(a) and abs(a) for a value with units.

    √a halves the dimensions

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Rounding to a step

    round, floor and ceil, with an optional step such as round(x, 5 mm).

    round(a ÷ s) × s

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Minimum, maximum, sum and average

    min, max, sum and avg of values of one kind.

    sum = Σ aᵢ in order; avg = sum ÷ n

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Trigonometry, logarithms and exponent

    sin, cos, tan, their inverses, atan2, ln, log10, exp and pow.

    sin, cos, tan of radians; asin, acos need |x| ≤ 1; ln, log10 need x > 0

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Reading an app parameter

    @Name: the app's parameter as a value in SI.

    value = parameter value × factor of its unit

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Values written to driven parameters

    What each drive would set in the app: the row's answer in the parameter's unit.

    value = row value ÷ factor of the unit; change when |current − value| > 1e-9 × max(1, |value|)

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Time-weighted average

    The average of a value that changes at events: queue length, units busy, work in process.

    average = (Σ vᵢ × Δtᵢ) ÷ (t − t₀)

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Mean, deviation and percentiles of observations

    Waiting time, service time and time in system: mean, standard deviation, minimum, maximum, P50 and P95.

    m ← m + (x − m) ÷ n; s ← s + (x − m_old)(x − m_new); sd = √(s ÷ (n − 1)); P = sorted[⌊p × k⌋]

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Event time

    When a scheduled event happens.

    t_event = t_now + delay

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Conveyor slots and timing

    How many items fit on a belt, the gap between items and the travel time.

    slots = max(1, ⌊length ÷ pitch + 1e-9⌋); headway = pitch ÷ speed; travel = length ÷ speed

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Shift edges

    When a server goes on and off shift.

    x = t − ⌊t ÷ period⌋ × period; on shift when a ≤ x < b

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Server utilization

    The share of time a server's units were busy, blocked, down and idle.

    utilization = average busy units ÷ units; idle = max(0, 1 − busy − blocked − down)

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Throughput

    Finished entities per unit of time at a sink.

    throughput = count ÷ (t − t₀)

    Flow simulation (discrete events) engine.js Every app
    64-bit

A calculation is added only after its source was read. Each statement can be checked in the code shown on its policy's page; the app repositories are public. When a release changes a calculation, its page says so until it has been read again.