Resources Numerical accuracy Calculation policy

Dimension text: units, rounding and tolerances

Writes the number a dimension shows: measurement scale, round-off, decimal places, degrees-minutes-seconds, alternate units and tolerance limits.

How the precision is kept

The text is made from the 64-bit measurement each time; the rounded text is never read back as the measurement. The value is multiplied by the measurement scale, rounded to the style's round-off with Math.round, and written with toFixed at 0 to 8 decimal places. Alternate units are computed from the primary value after round-off, with 25.4 as the default factor.

Open item. Use the same absolute value of the measurement scale for the limits of a tolerance as for the main text.

Calculations under this policy

11 calculations in 1 source file. The code is read from the released app when this page loads. It is the calculation people run, not a copy kept for this page.

dimstyle.js CadCraft 0.8.2 64-bit 11 calculations Reviewed

Tests bin/test-cadcraft-props.mjs, bin/test-cadcraft.mjs

Text for a measurement

64-bit

The main text, alternate-unit text and tolerance text a dimension style writes for a measured value.

m = round(v × |lfac|, rnd); alternate = round(m × altf, altrnd); limits: upper = base + tp, lower = base − tm

The alternate value is computed from m, the primary value after round-off, and not from the unrounded measurement. The main text uses |lfac|, but the base of a limits tolerance uses lfac without the absolute value, so a negative measurement scale gives limits of the opposite sign to the main text. The default alternate factor 25.4 is exact.

  • dimstyle.js
  • lines 147 to 178
  • CadCraft 0.8.2
/** The text a style writes for the measurement v (drawing units, or radians for angular). */
export function dimValue(e, v) {
  const st = full(e);
  const sep = st.dsep === ',' ? ',' : '.';
  let main;
  if (e.kind === 'angular') main = angle(v, { aunit: st.aunit, adec: st.adec, azin: st.azin, sep });
  else {
    const m = roundTo(v * (Number.isFinite(st.lfac) && st.lfac !== 0 ? Math.abs(st.lfac) : 1), st.rnd);
    main = linear(m, { unit: st.lunit, dec: st.dec, zin: st.zin, sep });
    if (st.alt) {
      const a = roundTo(m * (st.altf || 25.4), st.altrnd);
      main += ' [' + (st.apre || '') + linear(a, { unit: st.altu, dec: st.altd, zin: st.altz, sep }) + (st.asuf || '') + ']';
    }
  }
  const sym = e.kind === 'radius' ? 'R' : e.kind === 'diameter' ? '⌀' : '';
  main = (st.pre ? st.pre : sym) + main + (st.suf || '');
  let tol = null;
  const tfmt = (x) => (e.kind === 'angular' ? angle(x * D2R, { aunit: st.aunit, adec: st.tdec, azin: st.tzin, sep }) : linear(x, { unit: st.lunit, dec: st.tdec, zin: st.tzin, sep }));
  switch (st.tol) {
    case 'sym': tol = { kind: 'sym', text: '±' + tfmt(Math.abs(st.tp)) }; break;
    case 'dev': tol = { kind: 'dev', upper: '+' + tfmt(st.tp), lower: (st.tm > 0 ? '-' : st.tm < 0 ? '+' : '') + tfmt(Math.abs(st.tm)) }; break;
    case 'lim': {
      const base = e.kind === 'angular' ? v / D2R : roundTo(v * (st.lfac || 1), st.rnd);
      tol = { kind: 'lim', upper: tfmt(base + st.tp), lower: tfmt(base - st.tm) };
      main = '';
      break;
    }
    case 'basic': tol = { kind: 'basic' }; break;
    default: break;
  }
  return { main, tol };
}

Decimal number text

64-bit

Writes a value with a fixed number of decimal places and the style's zero suppression.

d = clamp(round(dec), 0, 8); |v| < 0.5 × 10⁻ᵈ → 0; text = v to d places

toFixed rounds the exact binary value, so a number that looks like a tie in decimal (2.675 at 2 places) goes the way its binary value lies. The number of places is capped at 8. The bitwise | 0 is applied to the place count, not to the value.

  • dimstyle.js
  • lines 107 to 113
  • CadCraft 0.8.2
function decimal(v, dec, { lead = false, trail = false, sep = '.' } = {}) {
  const d = Math.max(0, Math.min(8, Math.round(dec) | 0));
  let s = (Math.abs(v) < 0.5 * Math.pow(10, -d) ? 0 : v).toFixed(d);
  if (trail && s.includes('.')) s = s.replace(/0+$/, '').replace(/\.$/, '');
  if (lead) s = s.replace(/^(-?)0\./, '$1.');
  return sep === '.' ? s : s.replace('.', sep);
}

Round-off to a multiple

64-bit

Rounds a value to the nearest multiple of the style's round-off.

r > 0: Math.round(v ÷ r) × r; else v

Math.round rounds halves toward +∞, so −2.5 steps round to −2. The product n × r is a 64-bit product and can differ from the decimal multiple in the last digit (3 × 0.1 = 0.30000000000000004) before toFixed cuts it.

  • dimstyle.js
  • lines 114 to 115
  • CadCraft 0.8.2
/** Round to the nearest multiple of r (DIMRND); 0 = none. */
const roundTo = (v, r) => (r > 0 ? Math.round(v / r) * r : v);

Length text

64-bit

Writes a length as a decimal number or, for other unit formats, through the shared length formatter.

Decimal format goes to decimal() with the style's places and zero suppression bits (4 leading, 8 trailing). Other formats (feet and inches, fractions) are passed to formatLength in units.js with the same precision setting.

  • dimstyle.js
  • lines 117 to 121
  • CadCraft 0.8.2
function linear(v, { unit, dec, zin, sep }) {
  const code = lunitCode(unit || 2);
  if (code !== 2) return formatLength(v, { lunits: code, prec: dec });
  return decimal(v, dec, { lead: !!(zin & 4), trail: !!(zin & 8), sep });
}

Angle text

64-bit

Writes an angle in decimal degrees, degrees-minutes-seconds, grads or radians.

degrees = radians ÷ (π ÷ 180); minutes = (degrees − ⌊degrees⌋) × 60; seconds = (minutes − ⌊minutes⌋) × 60; grads = degrees ÷ 0.9

Subtracting the whole part is exact; each × 60 rounds once. With adec 1 or 2 minutes are rounded to whole minutes; with adec 3 or more seconds are rounded to adec − 4 decimals (whole seconds at 3 and 4), and 60 carries into the next unit. Grads divide by 0.9, which is not exactly representable, so the result can differ from degrees × 10 ÷ 9 in the last digit.

  • dimstyle.js
  • lines 123 to 143
  • CadCraft 0.8.2
function angle(rads, { aunit, adec, azin, sep }) {
  const lead = !!(azin & 1), trail = !!(azin & 2);
  const d = rads / D2R;
  switch (aunit | 0) {
    case 1: {   // degrees minutes seconds: adec 0 → d°, 1–2 → d°m', 3+ → d°m's"
      const n = Math.max(0, adec | 0), sgn = d < 0 ? '-' : '', a = Math.abs(d);
      if (n === 0) return sgn + Math.round(a) + '°';
      let deg = Math.floor(a), min = (a - deg) * 60;
      if (n <= 2) { min = Math.round(min); if (min === 60) { deg++; min = 0; } return sgn + deg + '°' + min + "'"; }
      let m = Math.floor(min), sec = (min - m) * 60;
      const sd = Math.max(0, n - 4);
      sec = +sec.toFixed(sd);
      if (sec >= 60) { sec = 0; m++; }
      if (m >= 60) { m = 0; deg++; }
      return sgn + deg + '°' + m + "'" + decimal(sec, sd, { trail, sep }) + '"';
    }
    case 2: return decimal(d / 0.9, adec, { lead, trail, sep }) + 'g';
    case 3: return decimal(rads, adec, { lead, trail, sep }) + 'r';
    default: return decimal(d, adec, { lead, trail, sep }) + '°';
  }
}

Millimeters in each text unit

Exact

The size of the units a dimension can write its value in.

mm = 1; cm = 10; m = 1000; km = 1 000 000

Whole numbers of millimeters, all exact in 64-bit.

  • dimstyle.js
  • line 67
  • CadCraft 0.8.2
export const TEXT_UNITS = Object.freeze({ mm: 1, cm: 10, m: 1000, km: 1e6 });

Measurement scale for a text unit

64-bit

Sets the scale that makes a dimension write its value in mm, cm, m or km.

lfac = mmPerUnit ÷ mm in the unit

One 64-bit division. The result is kept as the style's lfac (DIMLFAC), with the unit as the suffix and the format set to decimal.

  • dimstyle.js
  • lines 79 to 87
  • CadCraft 0.8.2
export function withTextUnit(st = {}, unit = '', mmPerUnit = 1) {
  const n = { ...st };
  if (!unit) { n.lfac = 1; if (unitSuffix(st)) n.suf = ''; return n; }
  if (!Object.hasOwn(TEXT_UNITS, unit) || !(mmPerUnit > 0)) return n;
  n.lfac = mmPerUnit / TEXT_UNITS[unit];
  n.suf = ' ' + unit;
  n.lunit = 2;
  return n;
}

Which text unit a style writes

64-bit

Recognizes a style's scale and suffix as mm, cm, m or km, as drawn, or custom.

|lfac − mmPerUnit ÷ unit| ≤ 1e-9 × max(1, lfac); as drawn: |lfac − 1| < 1e-12

The comparison with a named unit is relative (1e-9 of the scale, at least 1e-9); the test for a scale of 1 is absolute. A scale read from a file that is off by more than that is reported as custom and left as it is.

  • dimstyle.js
  • lines 70 to 77
  • CadCraft 0.8.2
export function textUnitOf(st = {}, mmPerUnit = 1) {
  const lfac = Number.isFinite(st.lfac) && st.lfac !== 0 ? Math.abs(st.lfac) : 1;
  const decimal = (st.lunit || 2) === 2;
  const u = unitSuffix(st);
  if (u && decimal && mmPerUnit > 0 && Math.abs(lfac - mmPerUnit / TEXT_UNITS[u]) <= 1e-9 * Math.max(1, lfac)) return u;
  if (!u && !st.suf && Math.abs(lfac - 1) < 1e-12) return '';
  return 'custom';
}

Millimeters in one drawing unit

64-bit

The size of a drawing unit, or of a sheet's paper unit, in millimeters.

paper in inches: 25.4; paper in mm: 1; model: mmPerDrawingUnit; unitless: 1 (assumed)

25.4 is the exact inch. The model value comes from units.js. A unitless drawing is read as millimeters and flagged as assumed.

  • dimstyle.js
  • lines 100 to 104
  • CadCraft 0.8.2
export function textUnitMm(meta = {}, layout = null) {
  if (layout?.paper) return { mm: layout.paper.unit === 'in' ? 25.4 : 1, assumed: false };   // a sheet: its paper's unit
  const mm = mmPerDrawingUnit(meta.lunits, meta.units);
  return mm ? { mm, assumed: false } : { mm: 1, assumed: true };
}

Refit after the drawing units change

64-bit

Gives dimensions that write in mm to km a new scale when the drawing's unit changes.

lfac′ = new mm per unit ÷ mm in the unit

The new scale is computed from the new unit size, not by multiplying the old scale by a ratio, so no rounding carries over from the old value. Nothing is done when the two unit sizes differ by less than 1e-12 (absolute).

  • dimstyle.js
  • lines 89 to 98
  • CadCraft 0.8.2
export function refitTextUnits(ents, oldMm, newMm) {
  const out = [];
  if (!(oldMm > 0) || !(newMm > 0) || Math.abs(oldMm - newMm) < 1e-12) return out;
  for (const e of ents) {
    if (e?.type !== 'dim' || !e.kind || e.kind === 'angular') continue;
    const u = textUnitOf(e.st || {}, oldMm);
    if (u && u !== 'custom') { const n = { ...e, st: withTextUnit(e.st || {}, u, newMm) }; delete n.block; out.push(n); }
  }
  return out;
}

Arrowhead shapes

64-bit

The lines, circle or filled outline of each arrowhead style at a tip, direction and size.

half-width = size ÷ 6; open 30°: tan(15°) × size; dot radius = size ÷ 4 (small: size ÷ 8), drawn with 16 sides; tick at 45°, half-length size ÷ 2

Corners are the tip moved along and across the arrow direction by cosine and sine of the 64-bit angle. The dot is a 16-sided filled polygon, not a circle; the origin mark is a true circle of radius size ÷ 4.

  • dimstyle.js
  • lines 198 to 225
  • CadCraft 0.8.2
/** Parts for one arrowhead at `tip`, pointing along `ang`, `size` long; `base` = layer/colour/role fields to copy. */
export function arrowParts(kind, tip, ang, size, base = {}) {
  const s = size, w = s / 6;
  const tri = (half) => [tip, at(tip, ang, s, half), at(tip, ang, s, -half)];
  switch (kind || '') {
    case 'NONE': return [];
    case 'CLOSEDBLANK': case 'CLOSED': {
      const [a, b, c] = tri(w);
      const out = [seg(a, b, base), seg(b, c, base), seg(c, a, base)];
      if (kind === 'CLOSED') out.push(seg(tip, at(tip, ang, s, 0), base));
      return out;
    }
    case 'OPEN': { const [a, b, c] = tri(w); return [seg(a, b, base), seg(a, c, base)]; }
    case 'OPEN30': { const h = Math.tan(15 * D2R) * s; const [a, b, c] = tri(h); return [seg(a, b, base), seg(a, c, base)]; }
    case 'OPEN90': { const [a, b, c] = tri(s); return [seg(a, b, base), seg(a, c, base)]; }
    case 'DOT': case 'DOTSMALL': {
      const r = kind === 'DOT' ? s / 4 : s / 8, n = 16, ring = [];
      for (let i = 0; i <= n; i++) ring.push(tip.x + r * Math.cos(i * 2 * Math.PI / n), tip.y + r * Math.sin(i * 2 * Math.PI / n));
      return [{ ...base, type: 'hatch', solid: true, loops: [ring] }];
    }
    case 'ORIGIN': return [{ ...base, type: 'circle', cx: tip.x, cy: tip.y, r: s / 4 }];
    case 'ARCHTICK': case 'OBLIQUE': {
      const a = ang + Math.PI / 4, h = s / 2;
      return [{ ...seg(P(tip.x - Math.cos(a) * h, tip.y - Math.sin(a) * h), P(tip.x + Math.cos(a) * h, tip.y + Math.sin(a) * h), base), ...(kind === 'ARCHTICK' ? { lw: 0.5 } : {}) }];
    }
    default: { const [a, b, c] = tri(w); return [{ ...base, type: 'hatch', solid: true, loops: [[a.x, a.y, b.x, b.y, c.x, c.y, a.x, a.y]] }]; }
  }
}

In detail

Method
value × |DIMLFAC|, rounded to a multiple of DIMRND, written with DIMDEC places; angles in decimal degrees, degrees-minutes-seconds, grads or radians; limits = value + upper and value − lower.
Number types
64-bit numbers; text by Number.toFixed. Unit factors: mm 1, cm 10, m 1000, km 1e6, inch 25.4.
Tolerances
A value smaller than half a unit in the last written place is written as 0. A measurement scale matches a text unit within 1e-9 × max(1, scale); it counts as 1 within 1e-12.
Limits
At most 8 decimal places are written. Seconds in degrees-minutes-seconds are written with at most adec − 4 decimals.

Approach: 64-bit, direct

Every value is a 64-bit number from input to result. Formulas are closed form or a direct solve, and every comparison uses a stated tolerance.

Every policy that takes this approach