Resources Numerical accuracy

Calculations

Every calculation under our calculation policies: what it works out, its formula and the precision it runs at. Open one to see its code as it is in the released app.

  • 6,917 calculations in 1,248 source files
  • 143 exact
  • 6,286 in 64-bit
  • 335 with a 32-bit step
  • Last reviewed

Calculations

  • Rotation vector from a matrix

    Axis times angle of a rotation, used for orientation error.

    θ = acos((tr R − 1) ÷ 2); v = θ ÷ (2 sin θ) × (R₃₂ − R₂₃, R₁₃ − R₃₁, R₂₁ − R₁₂)

    Rotations, transforms and small linear algebra (motion) math.js Every app
    64-bit
  • Rigid transforms

    Product, inverse and application of a rotation with a translation.

    A·B = (R_A R_B, p_A + R_A p_B); A⁻¹ = (R_Aᵀ, −R_Aᵀ p_A)

    Rotations, transforms and small linear algebra (motion) math.js Every app
    64-bit
  • Dense linear solve

    Solves the small systems of inverse kinematics and linkages.

    Gaussian elimination with partial pivoting

    Rotations, transforms and small linear algebra (motion) math.js Every app
    64-bit
  • Positive-definite test

    Whether a symmetric matrix, such as a mass matrix, is positive definite.

    Cholesky factorization succeeds

    Rotations, transforms and small linear algebra (motion) math.js Every app
    64-bit
  • Singular values

    The singular values of a Jacobian, used to detect singular poses and to count degrees of freedom.

    σᵢ = √(eigenvalues of J Jᵀ or Jᵀ J)

    Rotations, transforms and small linear algebra (motion) math.js Every app
    64-bit
  • Point to segment distance

    The distance from a point to a track center line or a polygon edge.

    t = clamp(((p − a) · (b − a)) ÷ |b − a|², 0, 1); d = |p − (a + t (b − a))|

    Board geometry and clearances geom.js Every app
    64-bit
  • Orientation of three points

    Which side of a line a point lies on.

    o = (b − a) × (c − a)

    Board geometry and clearances geom.js Every app
    64-bit
  • Segment crossing test

    Whether two segments cross or touch.

    cross when the orientation signs differ on both segments; touch when an orientation is 0 and the point lies in the other's box

    Board geometry and clearances geom.js Every app
    64-bit
  • Segment to segment distance

    The distance between two track center lines or edges.

    0 when they cross; else the least of four point-to-segment distances

    Board geometry and clearances geom.js Every app
    64-bit
  • Point in polygon

    Whether a point lies inside a zone, a pad or the board outline.

    even-odd rule along a ray in +x

    Board geometry and clearances geom.js Every app
    64-bit
  • Polygon area

    The signed area of a closed polygon.

    A = ½ Σ (xᵢ yᵢ₊₁ − xᵢ₊₁ yᵢ)

    Board geometry and clearances geom.js Every app
    64-bit
  • Polygon to polygon distance

    The distance between two closed shapes.

    0 when one first vertex is inside the other; else the least edge-to-edge distance

    Board geometry and clearances geom.js Every app
    64-bit
  • Gap between copper shapes

    The edge-to-edge clearance between two tracks, pads or zones.

    gap = max(0, center-line distance − r₁ − r₂)

    Board geometry and clearances geom.js Every app
    64-bit
  • Rotation of a point

    A pad or outline point turned by an angle in degrees.

    (x cos θ − y sin θ, x sin θ + y cos θ), θ = degrees × π ÷ 180

    Board geometry and clearances geom.js Every app
    64-bit
  • Pad position on the board

    Where a footprint pad sits on the board, with its shape for the clearance check.

    world = part position + R(part rotation) × (±x, y)

    Board geometry and clearances board.js Every app
    64-bit
  • Wrap into an integer type

    A number brought into the range of SINT, INT, DINT and the other integer types.

    v mod 2ⁿ, moved into the signed range

    Controller standard functions and integer types std.js Every app
    Exact
  • On-delay timer

    The elapsed time and output of a TON block.

    ET = min(now − t₀, PT); Q = ET ≥ PT

    Controller standard functions and integer types std.js Every app
    64-bit
  • Up counter

    The count and output of a CTU block.

    CV ← CV + 1 on a rising edge while CV < 32767; Q = CV ≥ PV

    Controller standard functions and integer types std.js Every app
    Exact
  • Round to nearest, ties to even

    The rounding used when a real is converted to an integer.

    nearest whole number; a tie goes to the even neighbor

    Controller standard functions and integer types std.js Every app
    Exact
  • Arithmetic functions

    ABS, SQRT, LN, LOG, EXP, the trigonometric functions, MIN, MAX, LIMIT, TRUNC, SHL and SHR.

    SQRT, LN, LOG, EXP, SIN and the rest from Math; SHL: x × 2ⁿ; SHR: ⌊x ÷ 2ⁿ⌋; TRUNC toward zero

    Controller standard functions and integer types std.js Every app
    64-bit
  • Type conversions

    REAL_TO_INT, INT_TO_REAL, BOOL_TO_INT, TIME_TO_REAL and the other A_TO_B functions.

    to an integer type: round to nearest (ties to even), then wrap; to a real type: unchanged

    Controller standard functions and integer types std.js Every app
    64-bit
  • Center, radius and sweep of a bulge arc

    The circle an arc segment lies on, from its two end points and its bulge.

    θ = 4·atan(b); k = (c ÷ 2) ÷ tan(θ ÷ 2); center = midpoint + k × left normal; r = |p − center|

    Arcs from bulges and three points geom.js
    • TakeoffCraft
    64-bit
  • Arc through three points

    The bulge of the arc that starts at one point, passes through a second and ends at a third.

    center = circumcenter(p, m, q); sweep = atan2(q − center) − atan2(p − center) ± 2π; b = tan(sweep ÷ 4)

    Arcs from bulges and three points geom.js
    • TakeoffCraft
    64-bit
  • Segment lengths of a run and their sum

    The length of each segment of an open run, and the run's total.

    L = Σ segLength(pᵢ₋₁, pᵢ, bᵢ₋₁)

    Arcs from bulges and three points geom.js
    • TakeoffCraft
    64-bit
  • Snap to the nearest 45° direction

    Moves a picked point onto the nearest of the eight 45° directions from the point before it, keeping its distance along that direction.

    u = round(cos, sin of k·45° to 12 decimals); p′ = base + u × ((p − base)·u)

    Arcs from bulges and three points geom.js
    • TakeoffCraft
    64-bit
  • Circle as two half-circle arcs

    Stores a drawn circle as two points on opposite sides of the center, each starting a half-circle arc.

    r = |edge − c|; points (c.x + r, c.y) and (c.x − r, c.y), b = 1 each

    Arcs from bulges and three points takeoffcraft.js
    • TakeoffCraft
    64-bit
  • Arc reshaped by dragging its middle

    Recomputes a segment's bulge when its arc handle is dragged.

    Arcs from bulges and three points takeoffcraft.js
    • TakeoffCraft
    64-bit
  • Length and area shown while drawing

    The length of the last segment, the run so far, the area and perimeter so far, or a circle's radius and area, shown in the status line as the pointer moves.

    length = L × k; area = A × k²; circle preview = π·r²

    Arcs from bulges and three points takeoffcraft.js
    • TakeoffCraft
    64-bit
  • Check distance between two points

    The real distance between two picked points, in feet-inches, decimal feet and meters.

    ft = |b − a| × k; m = ft × 0.3048

    Arcs from bulges and three points takeoffcraft.js
    • TakeoffCraft
    64-bit
  • Dropping repeated points when a mark is finished

    Removes a point that sits on the one before it, as a double-click adds.

    drop pᵢ when |pᵢ − previous kept point| ≤ 2 ÷ zoom

    Arcs from bulges and three points takeoffcraft.js
    • TakeoffCraft
    64-bit
  • Arcs broken into chords

    Replaces each arc of a run or outline by points on the arc, at most one step apart.

    k = max(2, ⌈|θ| ÷ step⌉); pⱼ = center + r·(cos tⱼ, sin tⱼ), tⱼ = a₀ + θ·j ÷ k

    Outline checks: inside, crossing, cut-outs geom.js
    • TakeoffCraft
    64-bit
  • Point inside an outline

    Whether a point lies inside a closed outline.

    Outline checks: inside, crossing, cut-outs geom.js
    • TakeoffCraft
    64-bit
  • Two segments cross or touch

    Whether two straight segments cross or touch.

    d₁ = (d − c) × (a − c), …; cross when d₁, d₂ lie beyond ±s on opposite sides and d₃, d₄ too

    Outline checks: inside, crossing, cut-outs geom.js
    • TakeoffCraft
    64-bit
  • Outline crosses itself

    Whether a closed outline crosses itself.

    Outline checks: inside, crossing, cut-outs geom.js
    • TakeoffCraft
    64-bit
  • Outline that cannot be measured

    Refuses an outline with too few points, one that crosses itself, or one with no area.

    Outline checks: inside, crossing, cut-outs geom.js
    • TakeoffCraft
    64-bit
  • Cut-out inside its area

    Checks that a cut-out lies fully inside the outline and does not overlap another cut-out.

    Outline checks: inside, crossing, cut-outs geom.js
    • TakeoffCraft
    64-bit
  • Round half away from zero

    Rounds a number to a given count of decimals.

    r = ⌊|v|·10ᵈ + 0.5 + 1e-9·max(1, |v|·10ᵈ)⌋ ÷ 10ᵈ

    Rounding for display and for money units.js
    • TakeoffCraft
    64-bit
  • Grouped fixed-point text

    Writes a number with a fixed count of decimals and thousands separators.

    Rounding for display and for money units.js
    • TakeoffCraft
    64-bit
  • Feet and inches to the nearest fraction

    Writes a length in feet as feet, inches and a fraction of an inch.

    n = round(|ft|·12·denom + 1e-9); feet = ⌊n ÷ (12·denom)⌋; inches = ⌊rem ÷ denom⌋

    Rounding for display and for money units.js
    • TakeoffCraft
    64-bit
  • Length in a chosen display unit

    Writes a length in feet-inches, decimal feet, inches, meters or millimeters.

    m = ft × 0.3048; mm = ft × 0.3048 × 1000

    Rounding for display and for money units.js
    • TakeoffCraft
    64-bit
  • Feet per sheet unit for a named scale

    The scale number for a preset such as 1/4 inch = 1 foot or 1:100.

    ftPerUnit = (userUnit ÷ 72) × ratio ÷ 12

    Sheet scale: presets, calibration and drawing units units.js
    • TakeoffCraft
    Exact
  • Scale from two points and a typed distance

    The scale that makes the distance between two picked points equal a typed real distance.

    ftPerUnit = realFt ÷ |b − a|

    Sheet scale: presets, calibration and drawing units units.js
    • TakeoffCraft
    64-bit
  • Named scale a number matches

    Finds the preset scale closest to a scale number, if any is close enough.

    err = |ftPerUnit − s| ÷ s ≤ 0.005

    Sheet scale: presets, calibration and drawing units units.js
    • TakeoffCraft
    64-bit
  • Feet per drawing unit by INSUNITS code

    The table that turns a drawing's unit code into feet per drawing unit.

    code 6: 1 ÷ 0.3048; code 21 as written: 1200 ÷ 3937

    Sheet scale: presets, calibration and drawing units cadcount.js
    • TakeoffCraft
    64-bit
  • The drawing's length unit

    Reads the INSUNITS code and looks up feet per drawing unit.

    Sheet scale: presets, calibration and drawing units cadcount.js
    • TakeoffCraft
    64-bit
  • Viewport transform, model to paper

    The matrix that places model space on the paper through a layout's viewport.

    k = h ÷ viewHeight; M = [k·c, k·s, −k·s, k·c, cx − k(c·vx − s·vy), cy − k(s·vx + c·vy)]

    Sheet scale: presets, calibration and drawing units cadset.js
    • TakeoffCraft
    64-bit
  • Sheet size and the drawing-to-page transform

    Puts a layout on a page in points, or fits model space to a page about 2000 units on its longest side.

    layout: page = (paper − min) × ptU; model: k = 2000 ÷ extent, page = k·x + (24 − k·minX)

    Sheet scale: presets, calibration and drawing units cadset.js
    • TakeoffCraft
    64-bit
  • Scale a drawing's sheet starts with

    Feet per page unit for a model sheet or a layout.

    model: f = ft ÷ k; layout: f = (1 ÷ ptU) ÷ s × ft, s = h ÷ viewHeight

    Sheet scale: presets, calibration and drawing units cadset.js
    • TakeoffCraft
    64-bit
  • Drawing scale taken over by the takeoff

    Gives each sheet of an opened drawing the scale the drawing itself states, when the sheet has none yet.

    Sheet scale: presets, calibration and drawing units takeoffcraft.js
    • TakeoffCraft
    64-bit
  • Calibration stored on the sheet

    Reads the typed distance, computes the scale and stores it with the two points and the text.

    Sheet scale: presets, calibration and drawing units takeoffcraft.js
    • TakeoffCraft
    64-bit

A calculation is added only after its source was read. Each statement can be checked in the code shown on its policy's page; the app repositories are public. When a release changes a calculation, its page says so until it has been read again.