Resources Numerical accuracy

Calculations

Every calculation under our calculation policies: what it works out, its formula and the precision it runs at. Open one to see its code as it is in the released app.

  • 6,917 calculations in 1,248 source files
  • 143 exact
  • 6,286 in 64-bit
  • 335 with a 32-bit step
  • Last reviewed

Calculations

  • Bit double, read

    A value stored as a two-bit code: a full 64-bit value, exactly 1.0 or exactly 0.0.

    Numbers in a DWG file, read and written bit for bit dwg/bits-read.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Double with default, read

    A value stored as the bytes in which it differs from a default value.

    Numbers in a DWG file, read and written bit for bit dwg/bits-read.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • 64-bit whole number, read

    An unsigned 64-bit whole number from two 32-bit halves.

    value = high × 2³² + low

    Numbers in a DWG file, read and written bit for bit dwg/bits-read.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Bit long long, read

    A whole number of up to seven bytes, the count given by three bits.

    value = Σ byteᵢ × 256ⁱ

    Numbers in a DWG file, read and written bit for bit dwg/bits-read.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Handle reference, read

    An object handle, absolute or as an offset from another handle.

    value = Σ byte × 256ᵏ; codes 6, 8, A, C: ref + 1, ref − 1, ref + value, ref − value

    Numbers in a DWG file, read and written bit for bit dwg/bits-read.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Modular char, read

    A signed whole number stored seven bits per byte.

    value = Σ (byte & 0x7F) × 2^(7i); the last byte carries 6 bits and the sign

    Numbers in a DWG file, read and written bit for bit dwg/bits-read.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Raw double, written

    A 64-bit value written as its eight bytes.

    Numbers in a DWG file, read and written bit for bit dwg/bits-write.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Bit double, written

    A value written as a two-bit code for exactly 0.0 or 1.0, or as a full 64-bit value.

    Numbers in a DWG file, read and written bit for bit dwg/bits-write.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Double with default, written

    A value written as a two-bit code when it equals its default, or in full.

    Numbers in a DWG file, read and written bit for bit dwg/bits-write.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Thickness, written

    A thickness written as one bit when it is zero, or as a bit double.

    Numbers in a DWG file, read and written bit for bit dwg/bits-write.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Bit long, written

    A 32-bit whole number in its short or full form.

    Numbers in a DWG file, read and written bit for bit dwg/bits-write.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    Exact
  • Bit short, written

    A 16-bit whole number in its short or full form.

    Numbers in a DWG file, read and written bit for bit dwg/bits-write.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    Exact
  • Bit long long, written

    A 64-bit whole number as a three-bit byte count and its bytes.

    Numbers in a DWG file, read and written bit for bit dwg/bits-write.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    Exact
  • Point, written

    The three coordinates of a point as bit doubles.

    Numbers in a DWG file, read and written bit for bit dwg/bits-write.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Handle bytes, written

    A handle as its bytes, most significant first.

    byte = v mod 256; v = ⌊v ÷ 256⌋

    Numbers in a DWG file, read and written bit for bit dwg/bits-write.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    Exact
  • Variant changed to another type

    The value of a variant as a string, a double, a single, an integer, a long or a boolean.

    integer = round(n); double = n

    Number conversion in LISP variants lisp/vla.js
    • CadCraft
    64-bit
  • Triangles of a polygon with holes

    The triangles of a face as triples of vertex indexes.

    Triangulation of faces with holes m3d/earcut.js
    • CadCraft
    • ModelCraft
    • TakeoffCraft
    64-bit
  • Orientation of three points

    Twice the signed area of a triangle, whose sign says which way three points turn.

    (q_y − p_y)(r_x − q_x) − (q_x − p_x)(r_y − q_y)

    Triangulation of faces with holes m3d/earcut.js
    • CadCraft
    • ModelCraft
    • TakeoffCraft
    64-bit
  • Direction of a ring

    Whether a ring of points runs clockwise or counterclockwise.

    Σ (x_j − x_i)(y_i + y_j)

    Triangulation of faces with holes m3d/earcut.js
    • CadCraft
    • ModelCraft
    • TakeoffCraft
    64-bit
  • Is a corner an ear

    Whether the triangle at a corner is convex and holds no other point of the polygon.

    Triangulation of faces with holes m3d/earcut.js
    • CadCraft
    • ModelCraft
    • TakeoffCraft
    64-bit
  • Point in a triangle

    Whether a point lies in or on a triangle.

    three comparisons (c − p) × (a − p) ≥ 0, each written as product ≥ product

    Triangulation of faces with holes m3d/earcut.js
    • CadCraft
    • ModelCraft
    • TakeoffCraft
    64-bit
  • Duplicate and collinear points removed

    Drops points that repeat their neighbor or lie exactly on the line between their neighbors.

    Triangulation of faces with holes m3d/earcut.js
    • CadCraft
    • ModelCraft
    • TakeoffCraft
    64-bit
  • Bridge from a hole to the outer ring

    The outer-ring point a hole is connected to, found by a ray to the left from the hole's leftmost point.

    x = p_x + (h_y − p_y)(n_x − p_x) ÷ (n_y − p_y); tan = |h_y − p_y| ÷ (h_x − p_x)

    Triangulation of faces with holes m3d/earcut.js
    • CadCraft
    • ModelCraft
    • TakeoffCraft
    64-bit
  • Do two edges cross

    Whether two segments intersect, used to keep a diagonal from crossing the polygon.

    Triangulation of faces with holes m3d/earcut.js
    • CadCraft
    • ModelCraft
    • TakeoffCraft
    64-bit
  • Is a diagonal's midpoint inside

    Whether the middle of a candidate diagonal lies inside the polygon.

    crossing count of a ray from ((a_x + b_x) ÷ 2, (a_y + b_y) ÷ 2)

    Triangulation of faces with holes m3d/earcut.js
    • CadCraft
    • ModelCraft
    • TakeoffCraft
    64-bit
  • Start and finish of each step

    When each step of each batch starts and finishes, and on which vessel.

    release = start + b × interval; t₀ = max(ready, vessel free); t₁ = t₀ + duration; wait = t₀ − ready

    Batch schedule and batch inventory process/batch.js
    • CadCraft
    • BimCraft
    64-bit
  • Makespan, utilization, cycle time and batches per day

    The length of the whole schedule, how busy each vessel is, the steady cycle time and the batches per day it allows.

    makespan = max finish − start; utilization = Σ(end − start) ÷ makespan; cycle = (work + clean × uses) ÷ vessels; batches per day = 24 ÷ cycle

    Batch schedule and batch inventory process/batch.js
    • CadCraft
    • BimCraft
    64-bit
  • Hours to whole days for the Gantt view

    The start and finish day of each task for the schedule view, which works in whole days.

    start day = day0 + ⌊start ÷ 24 + 1e-9⌋; finish day = day0 + max(⌊(finish − 1e-9) ÷ 24⌋, ⌊start ÷ 24 + 1e-9⌋)

    Batch schedule and batch inventory process/batch.js
    • CadCraft
    • BimCraft
    64-bit
  • Batch mole inventory and atom balance

    The moles of each component in each vessel after every operation, what was discharged, and how well the atoms balance.

    transfer: q = n × fraction; react: ξ = conversion × n_key ÷ |ν_key|, n ← n + ν × ξ; atom error = max |charged − held| ÷ charged

    Batch schedule and batch inventory process/batch.js
    • CadCraft
    • BimCraft
    64-bit
  • Channel counts and counts with spare

    How many analog and digital inputs and outputs the I/O list needs, and how many once the spare allowance is added.

    with spare = ⌈n × (1 + spare)⌉

    I/O channel counts with spare allowance process/controls.js
    • CadCraft
    • BimCraft
    64-bit
  • A stream's duty on one side of the pinch

    How much heat a stream has to give or take above the pinch, and below it.

    above, hot: Q = CP × (Ts − max(Tt, T_pinch)); above, cold: Q = CP × (Tt − max(Ts, T_pinch))

    Heat-exchanger network design process/hen.js
    • CadCraft
    • BimCraft
    64-bit
  • Choosing the next match

    Which hot and cold stream are matched next: streams at the pinch first, then the largest duty.

    score = 1e9 × (streams at the pinch) + min(remaining hot, remaining cold)

    Heat-exchanger network design process/hen.js
    • CadCraft
    • BimCraft
    64-bit
  • Duty and end temperatures of a match

    The heat a match exchanges, the temperatures of both streams at its two ends and whether it keeps the minimum temperature difference.

    Q = min(remaining hot, remaining cold); ΔT_hot = Q ÷ CP_hot; ΔT_cold = Q ÷ CP_cold; d₁ = |T_hot,₀ − T_cold,₀|; d₂ = |T_hot,₁ − T_cold,₁|

    Heat-exchanger network design process/hen.js
    • CadCraft
    • BimCraft
    64-bit
  • Heaters, coolers and totals

    The heating and cooling the matches leave over, and the number of units against the minimum.

    Q_H = Σ heater duties; Q_C = Σ cooler duties; units = matches + heaters + coolers

    Heat-exchanger network design process/hen.js
    • CadCraft
    • BimCraft
    64-bit
  • Numbers written on the grid drawing

    The CP, temperatures and duties shown as text on the network grid.

    label = value rounded to 1 decimal

    Heat-exchanger network design process/hen.js
    • CadCraft
    • BimCraft
    64-bit
  • Energy, cost and CO2 of a variant

    The totals a variant is judged by: heating, cooling and shaft power in kW, operating cost per hour and kg CO2 per hour.

    cost/h = (heat × p_heat + cool × p_cool) × 3600 ÷ 1e9 + (shaft ÷ 1e3) × p_power; CO2/h = (heat × 3600 ÷ 1e9) × f_heat + (shaft ÷ 1e3) × f_power

    Flowsheet variant metrics process/optimise.js
    • CadCraft
    • BimCraft
    64-bit
  • Proposed composition from lab results

    The mole fractions a sample's results give for a stream, for a person to accept or reject.

    w = % ÷ 100; w = ppm × 1e-6; w = (mg/L × 1e-3) ÷ ρ; nᵢ = wᵢ ÷ MWᵢ; zᵢ = nᵢ ÷ Σ n

    Lab results to stream composition process/env/lims.js
    • CadCraft
    • BimCraft
    64-bit
  • Calculated composition for the lab system

    A stream's composition as mass or mole fractions, percent or ppm, to send back as calculated values.

    wᵢ = zᵢ × MWᵢ ÷ Σ(z × MW); mass % = w × 100; ppm = w × 1e6

    Lab results to stream composition process/env/lims.js
    • CadCraft
    • BimCraft
    64-bit
  • Rational tag value

    A TIFF RATIONAL (two 32-bit integers) as one number.

    value = numerator ÷ denominator

    TIFF scan resolution raster/tiff.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • CivilCraft
    • PartCraft
    64-bit
  • Resolution in dots per inch

    The horizontal and vertical resolution of a page, in dots per inch.

    dpi = resolution × 2.54 (centimeter files); dpi = resolution (inch files)

    TIFF scan resolution raster/tiff.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • CivilCraft
    • PartCraft
    64-bit
  • Triangulate a polygon with holes

    The triangles of a polygon, as indexes into its point list.

    Polygon triangulation (ear clipping) usd/earcut.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Winding of a ring

    Whether a ring runs clockwise or counterclockwise.

    sum = Σ (xⱼ − xᵢ) × (yᵢ + yⱼ)

    Polygon triangulation (ear clipping) usd/earcut.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Orientation of three points

    On which side of the line through two points a third point lies.

    area = (q.y − p.y) × (r.x − q.x) − (q.x − p.x) × (r.y − q.y)

    Polygon triangulation (ear clipping) usd/earcut.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Removing duplicate and collinear points

    Drops points that repeat their neighbor or lie on the line between their neighbors.

    Polygon triangulation (ear clipping) usd/earcut.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Ear test

    Whether a corner can be cut off as a triangle: it is convex and no other point of the ring lies inside it.

    Polygon triangulation (ear clipping) usd/earcut.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Point in triangle

    Whether a point lies inside or on a triangle.

    three cross products of (corner − point) pairs, each ≥ 0

    Polygon triangulation (ear clipping) usd/earcut.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Bridge from a hole to the outer ring

    The point of the outer ring a hole is connected to.

    x = p.x + (h.y − p.y) × (p′.x − p.x) ÷ (p′.y − p.y); tan = |h.y − p.y| ÷ (h.x − p.x)

    Polygon triangulation (ear clipping) usd/earcut.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Do two segments cross

    Whether two edges intersect, including touching and collinear overlap.

    Polygon triangulation (ear clipping) usd/earcut.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Is the middle of a diagonal inside the polygon

    Whether a diagonal between two corners runs through the inside of the polygon.

    midpoint = ((a.x + b.x) ÷ 2, (a.y + b.y) ÷ 2); even-odd count of edges crossed by a ray

    Polygon triangulation (ear clipping) usd/earcut.js
    • CadCraft
    • BimCraft
    • ModelCraft
    • TakeoffCraft
    • CivilCraft
    • DetailCraft
    • PartCraft
    64-bit
  • Fire rating text as minutes

    A rating written as '2HR', '120 min', 'REI 120' or a bare number, as minutes.

    minutes = round(hours × 60)

    Assistant answers: fire ratings and room areas ask-tools.js
    • BimCraft
    64-bit

A calculation is added only after its source was read. Each statement can be checked in the code shown on its policy's page; the app repositories are public. When a release changes a calculation, its page says so until it has been read again.