Resources Numerical accuracy Calculation policy

Calculator results and export

Turns what a process calculator works out into the lines of its results table, its CSV export and the one figure it gives the status bar.

How the precision is kept

The engines return 64-bit numbers. The results table rounds each number to 6 significant digits when it builds the line, and the CSV export and the status-bar figure are read from those lines, so they carry 6 digits, not 15. A few calculators add their own arithmetic here: a net positive suction head margin, a relative difference between two energies, a lattice spacing.

Open item. Keep the full 64-bit value in each results line and round only when it is drawn, so the CSV export and the status-bar totals are not cut to 6 significant digits.

Calculations under this policy

7 calculations in 2 source files. The code is read from the released app when this page loads. It is the calculation people run, not a copy kept for this page.

process/calc.js CadCraft 0.8.2 64-bit 4 calculations Reviewed

Tests tools/process-test/test-draw4.mjs, tools/process-test/test-dec.mjs

Results table lines

64-bit

A result object as a flat list of named values for the calculator's table.

number → +v.toPrecision(6); list of numbers → count, first and last to 5 digits

Every number is rounded to 6 significant digits as the line is made, and the line keeps only the rounded value. A list of numbers is reduced to its length and its first and last entries.

  • process/calc.js
  • lines 15 to 33
  • CadCraft 0.8.2
export function flatten(obj, { max = 80, prefix = '' } = {}) {
  const out = [];
  const walk = (v, key, depth) => {
    if (out.length >= max || v === undefined || typeof v === 'function') return;
    if (v === null) { out.push({ k: key, v: '—' }); return; }
    if (typeof v === 'number') { out.push({ k: key, v: Number.isFinite(v) ? +v.toPrecision(6) : String(v) }); return; }
    if (typeof v === 'string' || typeof v === 'boolean') { out.push({ k: key, v: String(v).slice(0, 200) }); return; }
    if (Array.isArray(v)) {
      if (!v.length) return;
      if (v.every((x) => typeof x === 'number')) { out.push({ k: key, v: v.length + ' values, ' + (+v[0].toPrecision(5)) + ' … ' + (+v[v.length - 1].toPrecision(5)) }); return; }
      if (depth >= 3) return;
      v.slice(0, 12).forEach((x, i) => walk(x, key + '[' + (x && typeof x === 'object' && (x.id || x.name) ? (x.id || x.name) : i) + ']', depth + 1));
      return;
    }
    if (typeof v === 'object') { if (depth >= 3) return; for (const [k2, x] of Object.entries(v)) walk(x, key ? key + '.' + k2 : k2, depth + 1); }
  };
  walk(obj, prefix, 0);
  return out;
}

CSV export of the results

64-bit

The results table as comma-separated text.

Writes the lines as they are, so the exported numbers are the ones already rounded to 6 significant digits.

  • process/calc.js
  • line 34
  • CadCraft 0.8.2
export const toCsv = (lines) => 'quantity,value\n' + lines.map((l) => '"' + String(l.k).replace(/"/g, '""') + '","' + String(l.v).replace(/"/g, '""') + '"').join('\n') + '\n';

Figure for the status bar

64-bit

The yearly emissions, installed cost or campaign length a run adds to the running totals.

value = Number(the results line with that name)

The figure is read back from the results line, not from the engine's result, so the running total receives the value rounded to 6 significant digits.

  • process/calc.js
  • lines 361 to 368
  • CadCraft 0.8.2
/** CH.21 (EG.30 running totals): the one number a run gives the status bar, in the shell's base units: { measure: 'carbon'|'cost'|'time', value, currency?, label }. null = nothing to say. */
export function figuresOf(id, r) {
  const get = (k) => { const l = (r.lines || []).find((x) => x.k === k); const v = l && Number(l.v); return Number.isFinite(v) ? v : null; };
  if (id === 'emissions') { const v = get('perYearKg'); return v == null ? null : { measure: 'carbon', value: v, label: 'Process emissions per year' }; }
  if (id === 'cost') { const v = get('installed'); return v == null ? null : { measure: 'cost', value: v, currency: 'USD', label: 'Installed plant cost (screening estimate)' }; }
  if (id === 'batch') { const v = get('makespan'); return v == null ? null : { measure: 'time', value: v, label: 'Batch campaign length' }; }
  return null;
}

Molecular dynamics set-up

64-bit

The lattice spacing, cell count and cutoff of the Lennard-Jones liquid calculator.

a = ∛(4 ÷ ρ); n = min(max(2 | n_typed, 2), 6); r_c = min(2.5, n a ÷ 2 − 1e-6)

The cell count goes through a bitwise OR with 2, which truncates it to a whole number and also changes some values: a typed 4 becomes 6. The count is a number of cells, not a measured value, but the run then has more atoms than asked for.

  • process/calc.js
  • lines 326 to 327
  • CadCraft 0.8.2
const MD = await import('../molecular/md.js'), IO = await import('../molecular/io.js'); const aL = Math.cbrt(4 / i.rho), n = Math.min(Math.max(2 | i.n, 2), 6);
const q = +i.q || 0, atoms = MD.fccAtoms(n, aL).map((a, k) => (q ? { ...a, q: k % 2 ? q : -q } : a)), rc = Math.min(2.5, n * aL / 2 - 1e-6), L = n * aL;
process/calc4.js CadCraft 0.8.2 64-bit 3 calculations Reviewed

Tests None found by the review

Column outlet compositions

64-bit

The outlet flows and mole fractions the column calculator lists.

zᵢ → +zᵢ.toPrecision(5)

Mole fractions are rounded to 5 significant digits here, before the results table rounds again to 6.

  • process/calc4.js
  • lines 36 to 37
  • CadCraft 0.8.2
if (r.outs) r.outs.forEach((s, k) => { out['out' + (k + 1)] = { F_mol_s: s.F, z: Object.fromEntries(th.ids.map((id, j) => [id, +(s.z[j]).toPrecision(5)])) }; });
return { lines: lines(out), problems: fromEngine(r.problems).concat(r.converged ? [] : [prob('column2.converge', 'warning', 'The column did not converge in the iteration limit; the profile shown is not a solution.')]) };

Pump power and suction head margin

64-bit

The hydraulic power of a pump and the margin between the available and required net positive suction head.

power = ρ g Q H ÷ η; NPSH_r by straight-line interpolation between the maker's points; margin = NPSH_a − NPSH_r

The required head is interpolated linearly between the two points that bracket the flow; a flow outside the points gives no margin and a warning. The margin is a plain difference.

  • process/calc4.js
  • lines 107 to 112
  • CadCraft 0.8.2
if (i.eta > 0) out.hydraulicPower_W = i.rho * N.G * i.Q * head / i.eta;
if (Array.isArray(i.npshr) && i.npshr.length >= 2) {
  const p = i.npshr.slice().sort((x, y) => x[0] - y[0]); let r = null;
  for (let k = 1; k < p.length; k++) if (i.Q >= p[k - 1][0] && i.Q <= p[k][0]) r = p[k - 1][1] + (i.Q - p[k - 1][0]) / (p[k][0] - p[k - 1][0]) * (p[k][1] - p[k - 1][1]);
  if (r == null) problems.push(prob('npsh.range', 'warning', 'The flow is outside the NPSH-required points; no margin is given.'));
  else { out.npshr_m = r; out.margin_m = a - r; out.ratio = a / r; if (a < r) problems.push(prob('npsh.cavitation', 'error', 'NPSH available is below NPSH required: the pump will cavitate at this flow.')); else problems.push(prob('npsh.margin', 'info', 'NPSH available exceeds required by ' + (a - r).toFixed(2) + ' m; the margin you need is your project\'s decision.')); }

Mesh energy against the plain sum

64-bit

The relative difference between the particle-mesh and plain Ewald reciprocal energies.

|E_mesh − E_plain| ÷ |E_plain|

One subtraction and one division. The division is not guarded against a plain-sum energy of zero.

  • process/calc4.js
  • lines 198 to 199
  • CadCraft 0.8.2
const out = { energyMesh: p.E, energyPlain: plain.pe.coulRecip, relativeDifference: Math.abs(p.E - plain.pe.coulRecip) / Math.abs(plain.pe.coulRecip) };
const at = [], cons = []; let k = 0, m = Math.min(Math.max(i.dumbbells | 0 || 16, 1), 16);

In detail

Method
A recursive walk over the result object; numbers pass through toPrecision(6) and back to a number.
Number types
64-bit numbers, rounded to 6 significant digits in the lines; lists of numbers are summarized by their first and last value to 5 digits.
Limits
At most 80 lines per result, three levels deep, twelve entries per list. The molecular dynamics calculator combines the requested cell count with a bitwise OR.

Approach: 64-bit, direct

Every value is a 64-bit number from input to result. Formulas are closed form or a direct solve, and every comparison uses a stated tolerance.

Every policy that takes this approach