Resources Numerical accuracy

Calculations

Every calculation under our calculation policies: what it works out, its formula and the precision it runs at. Open one to see its code as it is in the released app.

  • 6,917 calculations in 1,248 source files
  • 143 exact
  • 6,286 in 64-bit
  • 335 with a 32-bit step
  • Last reviewed

Calculations

  • Distance from a point to a segment

    How far a sample point is from one segment of the other sheet.

    t = clamp((p − a)·d ÷ |d|², 0, 1); distance = hypot(p − (a + t·d))

    Drawing comparison by vector paths vecdiff.js
    • MarkupCraft
    64-bit
  • Grid of segments

    Files each segment under the grid cells its bounding box covers, to find nearby segments quickly.

    cells = (⌊x_max ÷ cell⌋ − ⌊x_min ÷ cell⌋ + 1) × (same in y); more than 4000 → not in the grid

    Drawing comparison by vector paths vecdiff.js
    • MarkupCraft
    64-bit
  • Parts of a sheet with no match

    The pieces of one sheet's segments that have no segment of the other sheet within the tolerance.

    n = max(1, ⌈length ÷ step⌉) samples; miss when no segment within tol

    Drawing comparison by vector paths vecdiff.js
    • MarkupCraft
    64-bit
  • Removed and added lines

    The lines of the old sheet that are not on the new one, and the other way round.

    Drawing comparison by vector paths vecdiff.js
    • MarkupCraft
    64-bit
  • Boxes round groups of changes

    Groups changed segments that are close together into boxes, for clouding and for the list of changes.

    merge boxes closer than gap; keep groups with Σ length ≥ minLen

    Drawing comparison by vector paths vecdiff.js
    • MarkupCraft
    64-bit
  • Size of a changed area in words

    The size of a changed area as text: inches on the sheet and, on a calibrated sheet, the real size.

    in = round(pt ÷ 72, 1 decimal); real = pt × mmPerPt, in m to 1 decimal from 1000 mm, else whole mm

    Drawing comparison by vector paths changes.js
    • MarkupCraft
    64-bit
  • Where on the sheet

    Names the part of the sheet a point is in, on a 3 × 3 grid (upper left, center, lower right and so on).

    column = ⌊(x − x₀) ÷ w × 3⌋; row = ⌊(y₁ − y) ÷ h × 3⌋, each clamped to 0…2

    Drawing comparison by vector paths changes.js
    • MarkupCraft
    64-bit
  • Amount of change in an area

    How many points of line were removed and added inside one changed area, and whether it counts as added, removed or modified.

    removed = Σ hypot over removed pieces in the box; modified when both ≥ 6 pt

    Drawing comparison by vector paths changes.js
    • MarkupCraft
    64-bit
  • Segments clipped to a region

    The parts of a sheet's segments that lie inside a task's region.

    Liang–Barsky: r = q ÷ p per box side; keep when t₁ − t₀ > 1e-9

    Drawing comparison by vector paths tasks.js
    • MarkupCraft
    64-bit
  • Region snapshot

    What a task's region link stores when it is made: the sheet's lines inside the region.

    keep the 300 longest clipped segments; each coordinate rounded to 0.1 pt

    Drawing comparison by vector paths tasks.js
    • MarkupCraft
    64-bit
  • Has the region changed

    Decides whether the drawing inside a task's region changed between the snapshot and the new revision.

    changed when removed + added ≥ 4 pt and (removed + added) ÷ stored length ≥ 0.02

    Drawing comparison by vector paths tasks.js
    • MarkupCraft
    64-bit
  • Has a model element changed

    Compares the stored state of a linked model element with its state now, number by number.

    same when |x − y| ≤ tol

    Drawing comparison by vector paths tasks.js
    • MarkupCraft
    64-bit
  • Removed and added line reported

    The amount of removed and added line shown after comparing two sheets by their lines.

    amount = round(Σ hypot(Δx, Δy)) in points

    Drawing comparison by vector paths markupcraft.js
    • MarkupCraft
    64-bit
  • Sheets ranked by change

    Ranks the sheet pairs of two sets by how much line changed and marks the ones that changed.

    len = round(Σ hypot over removed and added); changed when len ≥ 6

    Drawing comparison by vector paths markupcraft.js
    • MarkupCraft
    64-bit
  • Clouds round the changes

    Adds a cloud rectangle markup round each group of changes on every changed sheet.

    cloud box = cluster box grown by 8 pt on each side

    Drawing comparison by vector paths markupcraft.js
    • MarkupCraft
    64-bit
  • Product of two transforms

    Combines two affine transforms into one.

    p′ = p × M (row vectors); (m × n) applies m first

    Edits to a page's own content base-content.js
    • MarkupCraft
    64-bit
  • Inverse of a transform

    The inverse of the matrix an object is painted with.

    det = a·d − b·c; inverse = adjugate ÷ det

    Edits to a page's own content base-content.js
    • MarkupCraft
    64-bit
  • Path coordinates rewritten

    The new numbers of one path-building operator after a move, scale or turn.

    U = C × T × C⁻¹; (x′, y′) = (x, y) × U, written with at most 4 decimals

    Edits to a page's own content base-content.js
    • MarkupCraft
    64-bit
  • Matrix round a moved picture or form

    The cm matrix written before a picture or form XObject that was moved, scaled or turned.

    U = C × T × C⁻¹, six entries with at most 4 decimals

    Edits to a page's own content base-content.js
    • MarkupCraft
    64-bit
  • Is this a move at all

    Decides whether an edit's matrix changes anything.

    |M − identity| < 1e-9 on all six entries

    Edits to a page's own content base-content.js
    • MarkupCraft
    64-bit
  • Line width in the path's own units

    Converts a line width given in page units to the units of the path's own space.

    w = page width ÷ √|a·d − b·c|

    Edits to a page's own content base-content.js
    • MarkupCraft
    64-bit
  • Replacement picture fitted into the old one's box

    Scales a replacement picture to fit, centered, inside the box of the picture it replaces, keeping its own proportions.

    ia = W ÷ H; ba = box w ÷ box h; (sx, sy) = ia > ba ? (1, ba ÷ ia) : (ia ÷ ba, 1)

    Edits to a page's own content base-content.js
    • MarkupCraft
    64-bit
  • Advance of a text run as read

    How far each text-showing operator moves the text position when a page's content is read, kept so a later edit can leave the rest of the line in place.

    string: Σ (width ÷ 1000 × size + char spacing + word spacing on a single-byte space) × horizontal scale; number in TJ: −v ÷ 1000 × size × horizontal scale

    Edits to a page's own content base-content.js
    • MarkupCraft
    64-bit
  • Advance of a text run

    How far a string moves the text position, in unscaled text space.

    Σ (glyph width ÷ 1000 × size + char spacing + word spacing on a space) × horizontal scale

    Edits to a page's own content base-content.js
    • MarkupCraft
    64-bit
  • Padding after reworded text

    The adjustment written after a reworded run so the text that follows on the line does not move.

    k = −(old advance − new advance) × 1000 ÷ (size × horizontal scale)

    Edits to a page's own content base-content.js
    • MarkupCraft
    64-bit
  • Matrix of a moved text run

    The text matrix written before a text run that was moved.

    Tm′ = (Tm × C) × T × C⁻¹

    Edits to a page's own content base-content.js
    • MarkupCraft
    64-bit
  • Line position put back after a moved run

    Restores where the text line continues after a run on it was moved.

    Tm = line matrix; then advance by −(start + advance) × 1000 ÷ (size × horizontal scale)

    Edits to a page's own content base-content.js
    • MarkupCraft
    64-bit
  • A move added to an object's edits

    Adds a move, scale or turn to what an object already had.

    T′ = T_old × T_new

    Edits to a page's own content base-edit.js
    • MarkupCraft
    64-bit
  • A moved path point

    The new numbers of a path when one of its points or handles is dragged to a place on the page.

    u = p_page × (C × T)⁻¹; attached handles shift by u − u_old

    Edits to a page's own content base-edit.js
    • MarkupCraft
    64-bit
  • Drawn-in paths written to the page

    Writes paths that were drawn into the page, and the matrix of pasted content, as content stream text.

    Edits to a page's own content base-edit.js
    • MarkupCraft
    64-bit
  • A stored matrix accepted

    Checks a move matrix read from a file or autosave before it is used.

    |a·d − b·c| > 1e-9 and every |number| < 1e7

    Edits to a page's own content base-edit.js
    • MarkupCraft
    64-bit
  • Box of part of a text item

    The box on the sheet round some characters of one piece of the drawing's text.

    s = width × i ÷ n; box corners = origin + u·s + v·t, t from −0.22 to 0.9 of the size

    Text positions: search hits and the OCR text layer textsearch.js
    • MarkupCraft
    64-bit
  • Cloud round a search hit

    The box of the cloud markup placed round a found word.

    box = hit box grown by pad on each side

    Text positions: search hits and the OCR text layer textsearch.js
    • MarkupCraft
    64-bit
  • Picture scale for reading text

    The scale at which a scanned page is drawn before its text is read.

    s = min(dpi ÷ 72, maxSide ÷ max(w, h), √(maxPixels ÷ (w·h))), at least 0.05

    Text positions: search hits and the OCR text layer ocr.js
    • MarkupCraft
    64-bit
  • Matrix that places the text layer

    The matrix that puts the recognizer's text-only page onto the sheet, written into the saved PDF.

    M = viewport⁻¹ ∘ [W_px ÷ W_pt, 0, 0, −H_px ÷ H_pt, 0, H_px]

    Text positions: search hits and the OCR text layer ocr.js
    • MarkupCraft
    64-bit
  • Words placed on the sheet

    The position, direction and size of each word read from a scan, for Find and Select text.

    baseline y at x by linear interpolation along the line's baseline; size = height above the baseline ÷ 0.9

    Text positions: search hits and the OCR text layer ocr.js
    • MarkupCraft
    64-bit
  • Word count and mean confidence

    The number of words read on a page and their mean confidence, for the status line.

    conf = round(Σ confidence ÷ n)

    Text positions: search hits and the OCR text layer ocr.js
    • MarkupCraft
    64-bit
  • Significand width of a number

    How many bits of a number this device keeps.

    bits = the first n where 1 + 2⁻ⁿ = 1

    Device number check accuracy.js Every app
    Exact
  • Digits of a graphics float

    The number of significant digits stated for the graphics processor's float type.

    bits < 23: digits = max(1, ⌊(bits + 1) × log₁₀ 2⌋); bits ≥ 23: the fixed text "about 7"

    Device number check accuracy.js Every app
    64-bit
  • Size and position text

    A dimension, port size or port position as text in millimeters.

    text = round(x × 10⁴) ÷ 10⁴

    Library configurator sizes library-configurator.js Every app
    64-bit
  • Option value used for matching

    The text an option value is compared by when the page picks the variant that matches the selected options.

    numbers: round(v × 10⁴) ÷ 10⁴ as text; other values: trimmed text

    Library configurator sizes library-configurator.js Every app
    64-bit
  • Mesh buffers

    The item's box or mesh as geometry for drawing, turned from z up to y up.

    Library 3D preview library-3d.js Every app
    32-bit, display only
  • Camera position

    Where the camera sits as the view is turned and zoomed.

    position = target + dist × (sin φ sin θ, cos φ, sin φ cos θ)

    Library 3D preview library-3d.js Every app
    32-bit, display only
  • Variant body buffers

    The chosen variant's box or mesh as geometry for drawing.

    Library 3D preview library-configurator.js Every app
    32-bit, display only
  • Required and found values as text

    A limit or a measured value in words, such as ≤ 196.85 mm or 863.6–965.2 mm.

    text = round(x × 100) ÷ 100, then the unit

    Check report values problems.js Every app
    64-bit
  • Built-in unit factors

    The factor from each built-in unit of mass, time, force, pressure, energy, power, current, voltage and volume to SI.

    lb = 0.45359237 kg; lbf = 4.4482216152605 N; psi = 6894.757293168 Pa; psf = 47.88025898 Pa

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Constants and fallback lengths

    pi, e, standard gravity g0, and the length units used when no unit table is given.

    g0 = 9.80665 m/s²; in = 0.0254 m; ft = 0.3048 m; yd = 0.9144 m

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Unit lookup

    The SI factor and dimension of a unit name.

    deg = π ÷ 180; rad = 1

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Dimension comparison

    Whether two values have the same dimensions, so they can be added, compared or converted.

    |aᵢ − bᵢ| < 1e-9 for each of the five exponents

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit
  • Power

    a ^ b for a value with units.

    value = aᵇ; dimensions × b

    Calc sheet units, functions and driven parameters calc.js Every app
    64-bit

A calculation is added only after its source was read. Each statement can be checked in the code shown on its policy's page; the app repositories are public. When a release changes a calculation, its page says so until it has been read again.