Resources Numerical accuracy Calculation policy

Lengths as text: formats and typed lengths

Writes lengths and areas in the drawing's format (decimal, engineering, architectural, fractional, scientific) and reads lengths typed at the command line.

How the precision is kept

Reading combines feet, inches and fractions in 64-bit with one operation per part. A number typed with mm, cm, m, in or ft is converted through millimeters with a multiplication and a division, which can leave the last digit off; other unit marks use the shared table's single-rounding conversion. Writing rounds once to the format's step and splits with whole-number arithmetic.

Open item. Convert numbers typed with mm, cm, m, in or ft through the shared table's one-rounding conversion, as the other unit marks already are.

Calculations under this policy

9 calculations in 1 source file. The code is read from the released app when this page loads. It is the calculation people run, not a copy kept for this page.

units.js CadCraft 0.8.2 64-bit 9 calculations Reviewed

Tests bin/test-units.mjs, bin/test-cadcraft.mjs

Writing a length

64-bit

A length as text in one of the five formats.

engineering: r = round(|v| × 10^p) ÷ 10^p; feet = ⌊r ÷ 12 + 1e-12⌋; architectural: whole inches and a fraction of 1⁄2^p

Engineering rounds the inches to p decimals first, then takes whole feet as ⌊r ÷ 12 + 1e-12⌋, an absolute allowance. Architectural and fractional round once to 1⁄2^p in fraction(). Scientific uses toExponential(p).

  • units.js
  • lines 48 to 74
  • CadCraft 0.8.2
/** A length as text in a format. */
export function formatLength(v, { lunits = 2, prec = null } = {}) {
  if (!Number.isFinite(v)) return '';
  const code = lunitCode(lunits), p = precOf(code, prec);
  const neg = v < 0 ? '-' : '';
  switch (code) {
    case 1: return (Math.abs(v) < 1e-300 ? 0 : v).toExponential(p).toUpperCase().replace(/E([+-])(\d)$/, 'E$10$2');
    case 3: {
      const r = Math.round(Math.abs(v) * Math.pow(10, p)) / Math.pow(10, p);
      const ft = Math.floor(r / 12 + 1e-12), inch = r - ft * 12;
      if (!r) return '0\'-0"';
      return neg + ft + '\'-' + decimals(inch, p) + '"';
    }
    case 4: {
      const den = Math.pow(2, p), [whole, num, d] = fraction(v, den);
      const ft = Math.floor(whole / 12), inch = whole - ft * 12;
      if (!whole && !num) return '0\'-0"';
      return neg + ft + '\'-' + inch + (num ? ' ' + num + '/' + d : '') + '"';
    }
    case 5: {
      const den = Math.pow(2, p), [whole, num, d] = fraction(v, den);
      if (!whole && !num) return '0';
      return neg + (whole || !num ? String(whole) : '') + (num ? (whole ? ' ' : '') + num + '/' + d : '');
    }
    default: return neg + decimals(Math.abs(v), p);
  }
}

Whole part and fraction

64-bit

Splits a value into a whole number and a reduced fraction of a power of two.

n = round(|v| × den); whole = ⌊n ÷ den⌋; numerator = n − whole × den

One rounding to a whole number of 1/den, then whole-number arithmetic; the fraction is reduced by halving both parts.

  • units.js
  • lines 39 to 46
  • CadCraft 0.8.2
/** A whole part and a reduced fraction of den: [whole, num, den] (num 0 = none). */
function fraction(v, den) {
  let n = Math.round(Math.abs(v) * den);
  const whole = Math.floor(n / den);
  let num = n - whole * den, d = den;
  while (num && num % 2 === 0 && d % 2 === 0) { num /= 2; d /= 2; }
  return [whole, num, d];
}

Writing decimals

64-bit

A number with up to prec decimals, trailing zeros dropped.

|v| < 0.5 × 10^−prec → 0; else v.toFixed(prec)

Rounded by toFixed for the text only. Values below half a unit of the last decimal print as 0, which avoids a printed −0.

  • units.js
  • lines 34 to 37
  • CadCraft 0.8.2
function decimals(v, prec) {
  const a = Math.abs(v) < 0.5 * Math.pow(10, -prec) ? 0 : v;
  return Number(a.toFixed(prec)).toString();
}

Writing an area

64-bit

An area as text: square drawing units, or square inches and square feet in the feet-and-inches formats.

sq ft = sq in ÷ 144

The divisor 144 is exact. Feet-and-inches formats show at least 2 decimals; the fractional format shows 4 decimals.

  • units.js
  • lines 76 to 82
  • CadCraft 0.8.2
/** An area as text: square drawing units; for feet and inches, square inches and square feet. */
export function formatArea(a, { lunits = 2, prec = null } = {}) {
  const code = lunitCode(lunits), p = precOf(code, prec);
  if (isImperialFormat(code)) return t('units.area.imperial', { sqin: decimals(a, Math.max(p, 2)), sqft: decimals(a / 144, Math.max(p, 2)) });
  if (code === 1) return formatLength(a, { lunits: 1, prec: p });
  return decimals(a, code === 5 ? 4 : p);
}

Numbers and fractions typed

64-bit

Reads 6, 6.5, 6-1/2, 6 1/2 or 1/2 as a number.

whole + numerator ÷ denominator

One division and one addition; a zero denominator is refused.

  • units.js
  • lines 85 to 92
  • CadCraft 0.8.2
/** "6", "6.5", "6-1/2", "6 1/2", "1/2" → a number, else null. */
function mixed(t) {
  let m;
  if (NUM.test(t)) return +t;
  if ((m = t.match(/^(\d+)[- ](\d+)\/(\d+)$/)) && +m[3] > 0) return +m[1] + m[2] / m[3];
  if ((m = t.match(/^(\d+)\/(\d+)$/)) && +m[2] > 0) return m[1] / m[2];
  return null;
}

One typed length

64-bit

Reads a single length such as 12'6" or 6-1/2 into drawing units.

feet × 12 + inches, with the sign applied last

Feet-and-inch marks are read only when the drawing's format is engineering or architectural, where one drawing unit is one inch. Feet are multiplied by the whole number 12.

  • units.js
  • lines 192 to 216
  • CadCraft 0.8.2
function parseOne(raw, lunits = 2) {
  let s = String(raw ?? '').trim();
  if (!s) return null;
  let sign = 1;
  if (s[0] === '-' || s[0] === '+') { if (s[0] === '-') sign = -1; s = s.slice(1).trim(); }
  if (!s) return null;
  if (/['"]/.test(s)) {
    if (!isImperialFormat(lunitCode(lunits))) return null;
    let ft = 0, rest = s;
    const q = s.indexOf('\'');
    if (q >= 0) {
      const f = mixed(s.slice(0, q).trim());
      if (f === null) return null;
      ft = f;
      rest = s.slice(q + 1).trim().replace(/^-/, '');
    }
    rest = rest.replace(/"$/, '').trim();
    if (/['"]/.test(rest)) return null;
    let inch = 0;
    if (rest) { const i = mixed(rest); if (i === null) return null; inch = i; }
    return sign * (ft * 12 + inch);
  }
  const v = mixed(s);
  return v === null ? null : sign * v;
}

Millimeters per drawing unit

Exact

The size of one drawing unit in millimeters, from the drawing's units.

inch 25.4; foot 304.8; yard 914.4; mile 1 609 344; micron 0.001

The values are the defined sizes, not rounded ones. A drawing with no units in a feet-and-inches format counts as inches; otherwise no unit means no conversion.

  • units.js
  • lines 107 to 113
  • CadCraft 0.8.2
const UNIT_MM = { millimeters: 1, centimeters: 10, meters: 1000, inches: 25.4, feet: 304.8, decimeters: 100, kilometers: 1e6, miles: 1609344, yards: 914.4, microns: 0.001 };
const SUFFIX_MM = { mm: 1, cm: 10, m: 1000, in: 25.4, inch: 25.4, inches: 25.4, ft: 304.8, foot: 304.8, feet: 304.8 };
/** Millimetres in one drawing unit, or null when the drawing does not say (a unitless feet-and-inches drawing: inches). */
export function mmPerDrawingUnit(lunits = 2, units = '') {
  const u = UNIT_MM[String(units || '').toLowerCase()];
  return u || (isImperialFormat(lunitCode(lunits)) ? 25.4 : null);
}

A length typed with a unit

64-bit

Converts a number with a unit mark, such as 450mm or 6in, into drawing units.

mm, cm, m, in, ft: value × mm per typed unit ÷ mm per drawing unit; other marks: parseTo into the drawing's unit

For mm, cm, m, in and ft the value is multiplied and then divided, two rounded operations: 6in in a drawing in feet gives 0.4999999999999999, not 0.5. Every other mark goes through parseTo of the unit table, which converts with one rounding.

  • units.js
  • lines 115 to 127
  • CadCraft 0.8.2
/** One length (no arithmetic) → { value, unit } for lencalc.js, or null. */
function lengthAtom(s, lunits, units) {
  const v = parseOne(s, lunits);
  if (v !== null) return { value: v, unit: /['"]/.test(s) };
  const m = /^((?:\d+(?:\.\d*)?|\.\d+)(?:e[-+]?\d+)?)\s*(mm|cm|m|in|inch|inches|ft|foot|feet)$/i.exec(String(s).trim());
  const per = mmPerDrawingUnit(lunits, units);
  if (m) return per ? { value: m[1] * SUFFIX_MM[m[2].toLowerCase()] / per, unit: true } : null;
  // every other length mark of the shared table (0.7 UNITS): 3 yd, 2 ch, 10 US ft, 1.5 km, 400 nm — when the drawing's units are known
  if (!per || /['"]/.test(s)) return null;
  const to = DRAWING_ID[String(units || '').toLowerCase()];   // straight into the drawing's unit: one rounding
  const r = parseTo(s, 'length', to || 'mm');
  return r.ok ? { value: to ? r.value : r.value / per, unit: true } : null;
}

Writing an angle

64-bit

An angle in degrees as the drawing's angle format writes it.

decimal: deg.toFixed(4)

Decimal degrees are shown to 4 decimals and an angle under 1e-10 degrees prints as 0. Other formats are written by formatAngle of angles.js with the drawing's own precision.

  • units.js
  • lines 165 to 171
  • CadCraft 0.8.2
/** An angle in degrees as the drawing's angle format writes it: "45°" (decimal, unchanged from before), 45°23'30", N44°36'30"E. */
export function fmtAngle(meta, deg) {
  const u = aunitCode(meta?.aunits ?? 0);
  if (!u) return (Math.abs(deg) < 1e-10 ? '0' : Number(deg.toFixed(4)).toString()) + '°';
  const p = Number.isFinite(meta?.auprec) ? meta.auprec : 4;
  return formatAngle(deg, u, p);
}

In detail

Method
Formats follow the DXF length-format codes 1 to 5; fractions to 1⁄2^prec.
Number types
64-bit numbers.
Tolerances
Engineering format adds 1e-12 (absolute) before taking whole feet. Decimal output prints values under half a unit of the last decimal as 0. Angles under 1e-10 degrees print as 0.
Limits
Text output carries at most 8 decimals or 1⁄256.

Approach: 64-bit, direct

Every value is a 64-bit number from input to result. Formulas are closed form or a direct solve, and every comparison uses a stated tolerance.

Every policy that takes this approach