Resources Numerical accuracy Calculation policy

Units and tolerance read from a 3DM file

The size of one file unit in meters and the modeling tolerance of an opened 3DM file.

How the precision is kept

A table maps the file's unit system number to meters per unit. Most entries are exact; the parsec entry is written to 8 digits. The values are read as 64-bit numbers from the file.

Calculations under this policy

2 calculations in 1 source file. The code is read from the released app when this page loads. It is the calculation people run, not a copy kept for this page.

io/3dm/read.js ModelCraft 0.8.1 64-bit 2 calculations Reviewed Nov 6, 2026

Tests None found by the review

Meters per file unit

64-bit

The conversion factor for each unit system a 3DM file can name.

meters = file value × factor

Inch 0.0254, foot 0.3048, yard 0.9144, mile 1609.344 and the metric steps are exact. Point and pica are written as 0.0254 ÷ 72 and 0.0254 ÷ 6. The parsec is written 3.0856776e16, which is rounded; astronomical unit and light year are given to their defined values.

io/3dm/read.js
lines 24 to 30
ModelCraft 0.8.1

export const UNITS = {
  0: ['none', 1], 1: ['um', 1e-6], 2: ['mm', 1e-3], 3: ['cm', 1e-2], 4: ['m', 1], 5: ['km', 1e3], 6: ['uin', 2.54e-8], 7: ['mil', 2.54e-5],
  8: ['in', 0.0254], 9: ['ft', 0.3048], 10: ['mi', 1609.344], 12: ['A', 1e-10], 13: ['nm', 1e-9], 14: ['dm', 0.1], 15: ['dam', 10],
  16: ['hm', 100], 17: ['Mm', 1e6], 18: ['Gm', 1e9], 19: ['yd', 0.9144], 20: ['pt', 0.0254 / 72], 21: ['pc', 0.0254 / 6],
  22: ['nmi', 1852], 23: ['au', 1.495978707e11], 24: ['ly', 9.4607304725808e15], 25: ['pc', 3.0856776e16],
};

Units and tolerance from the file settings

64-bit

Reads the unit system, custom meters-per-unit and the tolerance from a file's settings record.

The tolerance and the custom meters-per-unit are read as doubles and used only when finite and positive. If the record cannot be read the model stays in millimeters with tolerance 0.001.

io/3dm/read.js
lines 86 to 109
ModelCraft 0.8.1

function settings(r, out) {
  while (!r.atEnd()) {
    const c = r.chunk();
    if (c.short) continue;
    const s = r.sub(c);
    if (c.tc !== TC.UNITSANDTOLS) continue;
    try {
      const v = s.i32();
      if (v < 100 || v >= 200) continue;
      const system = s.u32();
      const tol = s.f64();
      s.f64(); s.f64();
      let metres = UNITS[system] ? UNITS[system][1] : 1e-3;
      let name = UNITS[system] ? UNITS[system][0] : 'mm';
      if (v >= 102) {
        s.i32(); s.i32();
        const mpu = s.f64();
        if (system === 11 && mpu > 0 && Number.isFinite(mpu)) { metres = mpu; name = 'custom'; }
      } else if (system === 11) { name = 'mm'; metres = 1e-3; }
      out.units = { system, name, metres };
      if (tol > 0 && Number.isFinite(tol)) out.tolerance = tol;
    } catch { /* keep millimetres */ }
  }
}

In detail

Method: Table lookup by the unit system number; a custom system uses the meters-per-unit the file stores.
Number types: 64-bit numbers read with f64 from the file.
Tolerances: The default tolerance is 0.001 file units; a tolerance from the file is used when it is finite and above 0.
Limits: A unit system not in the table, or a missing settings record, is read as millimeters. The parsec factor 3.0856776e16 is cut to 8 digits (relative error about 6e-9).

Approach: 64-bit, stated limit

The arithmetic is 64-bit, but the method or the data sets the accuracy: a solver tolerance, a mesh, a step size, a datum shift, a table. The limit is stated where the result is shown.

Every policy that takes this approach