Resources Numerical accuracy

Calculations

Every calculation under our calculation policies: what it works out, its formula and the precision it runs at. Open one to see its code as it is in the released app.

  • 6,917 calculations in 1,248 source files
  • 143 exact
  • 6,286 in 64-bit
  • 335 with a 32-bit step
  • Last reviewed

Calculations

  • Newton iteration

    Finds the state that satisfies the equations at one time.

    J_ij = (F_i(x + h e_j) − F_i(x − h e_j)) ÷ 2h, h = 1e-7 × max(|x_j|, 1); stop when |Δxᵢ| ≤ rtol |xᵢ| + atol + 1e-11 × max(1, largest |x|)

    Lumped system models solve.js Every app
    64-bit
  • Operating point

    The steady state, or the state set by the given initial values.

    one Newton solve from zero

    Lumped system models solve.js Every app
    64-bit
  • One time step

    The state one step later.

    trapezoid: (2 ÷ h)(S − S₀) − Ṡ₀ + g = 0; backward Euler: (1 ÷ h)(S − S₀) + g = 0

    Lumped system models solve.js Every app
    64-bit
  • Step-size control

    Chooses the time step from an estimate of the error, when adaptive stepping is on.

    err = max |x(two half steps) − x(one step)| ÷ (rtol |x| + atol), ÷ 3 for the trapezoid; h ← h × clamp(0.9 err^(−1/3), 0.2, 5)

    Lumped system models solve.js Every app
    64-bit
  • Value between stored times

    A node value at a time between two computed points.

    y = y₀ + (y₁ − y₀)(t − t₀) ÷ (t₁ − t₀)

    Lumped system models solve.js Every app
    64-bit
  • Parameter expressions

    The value of a parameter written as an expression, such as 2*3.3e3.

    numbers, parameters, + − × ÷ ^, brackets, pi

    Lumped system models modelica.js Every app
    64-bit
  • Thermal conductance to resistance

    A thermal conductor from a model file as the engine's thermal resistance.

    R = 1 ÷ G

    Lumped system models modelica.js Every app
    64-bit
  • Center, radius and sweep of a bulge arc

    The circle an arc segment lies on, from its two end points and its bulge.

    θ = 4·atan(b); k = (c ÷ 2) ÷ tan(θ ÷ 2); center = midpoint + k × left normal; r = |p − center|

    Arcs from bulges and three points geom.js
    • TakeoffCraft
    64-bit
  • Arc through three points

    The bulge of the arc that starts at one point, passes through a second and ends at a third.

    center = circumcenter(p, m, q); sweep = atan2(q − center) − atan2(p − center) ± 2π; b = tan(sweep ÷ 4)

    Arcs from bulges and three points geom.js
    • TakeoffCraft
    64-bit
  • Segment lengths of a run and their sum

    The length of each segment of an open run, and the run's total.

    L = Σ segLength(pᵢ₋₁, pᵢ, bᵢ₋₁)

    Arcs from bulges and three points geom.js
    • TakeoffCraft
    64-bit
  • Snap to the nearest 45° direction

    Moves a picked point onto the nearest of the eight 45° directions from the point before it, keeping its distance along that direction.

    u = round(cos, sin of k·45° to 12 decimals); p′ = base + u × ((p − base)·u)

    Arcs from bulges and three points geom.js
    • TakeoffCraft
    64-bit
  • Circle as two half-circle arcs

    Stores a drawn circle as two points on opposite sides of the center, each starting a half-circle arc.

    r = |edge − c|; points (c.x + r, c.y) and (c.x − r, c.y), b = 1 each

    Arcs from bulges and three points takeoffcraft.js
    • TakeoffCraft
    64-bit
  • Arc reshaped by dragging its middle

    Recomputes a segment's bulge when its arc handle is dragged.

    Arcs from bulges and three points takeoffcraft.js
    • TakeoffCraft
    64-bit
  • Length and area shown while drawing

    The length of the last segment, the run so far, the area and perimeter so far, or a circle's radius and area, shown in the status line as the pointer moves.

    length = L × k; area = A × k²; circle preview = π·r²

    Arcs from bulges and three points takeoffcraft.js
    • TakeoffCraft
    64-bit
  • Check distance between two points

    The real distance between two picked points, in feet-inches, decimal feet and meters.

    ft = |b − a| × k; m = ft × 0.3048

    Arcs from bulges and three points takeoffcraft.js
    • TakeoffCraft
    64-bit
  • Dropping repeated points when a mark is finished

    Removes a point that sits on the one before it, as a double-click adds.

    drop pᵢ when |pᵢ − previous kept point| ≤ 2 ÷ zoom

    Arcs from bulges and three points takeoffcraft.js
    • TakeoffCraft
    64-bit
  • Arcs broken into chords

    Replaces each arc of a run or outline by points on the arc, at most one step apart.

    k = max(2, ⌈|θ| ÷ step⌉); pⱼ = center + r·(cos tⱼ, sin tⱼ), tⱼ = a₀ + θ·j ÷ k

    Outline checks: inside, crossing, cut-outs geom.js
    • TakeoffCraft
    64-bit
  • Point inside an outline

    Whether a point lies inside a closed outline.

    Outline checks: inside, crossing, cut-outs geom.js
    • TakeoffCraft
    64-bit
  • Two segments cross or touch

    Whether two straight segments cross or touch.

    d₁ = (d − c) × (a − c), …; cross when d₁, d₂ lie beyond ±s on opposite sides and d₃, d₄ too

    Outline checks: inside, crossing, cut-outs geom.js
    • TakeoffCraft
    64-bit
  • Outline crosses itself

    Whether a closed outline crosses itself.

    Outline checks: inside, crossing, cut-outs geom.js
    • TakeoffCraft
    64-bit
  • Outline that cannot be measured

    Refuses an outline with too few points, one that crosses itself, or one with no area.

    Outline checks: inside, crossing, cut-outs geom.js
    • TakeoffCraft
    64-bit
  • Cut-out inside its area

    Checks that a cut-out lies fully inside the outline and does not overlap another cut-out.

    Outline checks: inside, crossing, cut-outs geom.js
    • TakeoffCraft
    64-bit
  • Convex outline test

    Whether a closed outline has no inward corner.

    Back-outs clipped out of areas geom.js
    • TakeoffCraft
    64-bit
  • Polygon clipped to a convex polygon

    The part of a polygon that lies inside a convex polygon.

    t = d₁ ÷ (d₁ − d₂); corner = p + (q − p)·t

    Back-outs clipped out of areas geom.js
    • TakeoffCraft
    64-bit
  • Area a back-out covers

    How much of an area mark (outline minus cut-outs) a convex back-out covers, in square PDF units.

    overlap = max(0, A(outline ∩ b) − Σ A(cut-out ∩ b))

    Back-outs clipped out of areas geom.js
    • TakeoffCraft
    64-bit
  • Net area shown for a mark

    The area in a mark's properties after its back-outs are taken off.

    net = max(0, area − cut); plan area = max(0, plan − cut ÷ factor)

    Back-outs clipped out of areas takeoffcraft.js
    • TakeoffCraft
    64-bit
  • Typed length to feet

    Normalizes what was typed and reads it as one length, or as arithmetic over lengths.

    Typed lengths and item units units.js
    • TakeoffCraft
    64-bit
  • One length with a metric unit, or feet and inches

    Reads one normalized length into feet.

    ft = meters ÷ 0.3048

    Typed lengths and item units units.js
    • TakeoffCraft
    Exact
  • Numbers and fractions in a typed length

    Splits the text into numbers, fractions and foot or inch marks.

    fraction = a ÷ b (integers)

    Typed lengths and item units units.js
    • TakeoffCraft
    Exact
  • Feet and inches combined

    Combines a feet part and an inches part (whole number plus fraction) into feet.

    ft = feet + (inches + fraction) ÷ 12

    Typed lengths and item units units.js
    • TakeoffCraft
    Exact
  • Bare number in the default unit to feet

    Converts a number in feet, inches, meters, centimeters or millimeters to feet.

    ft = v ÷ 12; ft = v ÷ 0.3048; ft = v ÷ 1000 ÷ 0.3048

    Typed lengths and item units units.js
    • TakeoffCraft
    Exact
  • Item unit factors

    The factor from feet or square feet to each unit an item can be reported in.

    Typed lengths and item units units.js
    • TakeoffCraft
    Exact
  • Feet and square feet into an item's unit

    Converts a length in feet or an area in square feet to the unit that goes with an item.

    m = ft × 0.3048; area = sf × perSqFt

    Typed lengths and item units units.js
    • TakeoffCraft
    Exact
  • Round half away from zero

    Rounds a number to a given count of decimals.

    r = ⌊|v|·10ᵈ + 0.5 + 1e-9·max(1, |v|·10ᵈ)⌋ ÷ 10ᵈ

    Rounding for display and for money units.js
    • TakeoffCraft
    64-bit
  • Grouped fixed-point text

    Writes a number with a fixed count of decimals and thousands separators.

    Rounding for display and for money units.js
    • TakeoffCraft
    64-bit
  • Feet and inches to the nearest fraction

    Writes a length in feet as feet, inches and a fraction of an inch.

    n = round(|ft|·12·denom + 1e-9); feet = ⌊n ÷ (12·denom)⌋; inches = ⌊rem ÷ denom⌋

    Rounding for display and for money units.js
    • TakeoffCraft
    64-bit
  • Length in a chosen display unit

    Writes a length in feet-inches, decimal feet, inches, meters or millimeters.

    m = ft × 0.3048; mm = ft × 0.3048 × 1000

    Rounding for display and for money units.js
    • TakeoffCraft
    64-bit
  • Feet per sheet unit for a named scale

    The scale number for a preset such as 1/4 inch = 1 foot or 1:100.

    ftPerUnit = (userUnit ÷ 72) × ratio ÷ 12

    Sheet scale: presets, calibration and drawing units units.js
    • TakeoffCraft
    Exact
  • Scale from two points and a typed distance

    The scale that makes the distance between two picked points equal a typed real distance.

    ftPerUnit = realFt ÷ |b − a|

    Sheet scale: presets, calibration and drawing units units.js
    • TakeoffCraft
    64-bit
  • Named scale a number matches

    Finds the preset scale closest to a scale number, if any is close enough.

    err = |ftPerUnit − s| ÷ s ≤ 0.005

    Sheet scale: presets, calibration and drawing units units.js
    • TakeoffCraft
    64-bit
  • Feet per drawing unit by INSUNITS code

    The table that turns a drawing's unit code into feet per drawing unit.

    code 6: 1 ÷ 0.3048; code 21 as written: 1200 ÷ 3937

    Sheet scale: presets, calibration and drawing units cadcount.js
    • TakeoffCraft
    64-bit
  • The drawing's length unit

    Reads the INSUNITS code and looks up feet per drawing unit.

    Sheet scale: presets, calibration and drawing units cadcount.js
    • TakeoffCraft
    64-bit
  • Viewport transform, model to paper

    The matrix that places model space on the paper through a layout's viewport.

    k = h ÷ viewHeight; M = [k·c, k·s, −k·s, k·c, cx − k(c·vx − s·vy), cy − k(s·vx + c·vy)]

    Sheet scale: presets, calibration and drawing units cadset.js
    • TakeoffCraft
    64-bit
  • Sheet size and the drawing-to-page transform

    Puts a layout on a page in points, or fits model space to a page about 2000 units on its longest side.

    layout: page = (paper − min) × ptU; model: k = 2000 ÷ extent, page = k·x + (24 − k·minX)

    Sheet scale: presets, calibration and drawing units cadset.js
    • TakeoffCraft
    64-bit
  • Scale a drawing's sheet starts with

    Feet per page unit for a model sheet or a layout.

    model: f = ft ÷ k; layout: f = (1 ÷ ptU) ÷ s × ft, s = h ÷ viewHeight

    Sheet scale: presets, calibration and drawing units cadset.js
    • TakeoffCraft
    64-bit
  • Drawing scale taken over by the takeoff

    Gives each sheet of an opened drawing the scale the drawing itself states, when the sheet has none yet.

    Sheet scale: presets, calibration and drawing units takeoffcraft.js
    • TakeoffCraft
    64-bit
  • Calibration stored on the sheet

    Reads the typed distance, computes the scale and stores it with the two points and the text.

    Sheet scale: presets, calibration and drawing units takeoffcraft.js
    • TakeoffCraft
    64-bit
  • Typed unit rate

    Reads a rate such as $1,234.5 into whole 1 ÷ 10000 dollars.

    rate4 = round(roundTo(v, 4) × 10000)

    Rates, percentages and extended amounts units.js
    • TakeoffCraft
    Exact
  • Typed percentage

    Reads a percentage such as 7.5 % into whole basis points.

    bp = round(roundTo(v, 2) × 100)

    Rates, percentages and extended amounts units.js
    • TakeoffCraft
    Exact
  • Quantity × rate to the cent

    The extended amount of a line in whole cents.

    cents = round(qty × rate4 ÷ 100)

    Rates, percentages and extended amounts units.js
    • TakeoffCraft
    64-bit

A calculation is added only after its source was read. Each statement can be checked in the code shown on its policy's page; the app repositories are public. When a release changes a calculation, its page says so until it has been read again.