Resources Numerical accuracy Calculation policy
The coordinates, normals and placement matrices written when a model is exported as an OBJ or a Collada (.dae) text file.
The geometry is computed in 64-bit, but the numbers are rounded when they are written as text: OBJ to 6 decimals of a meter, Collada to 12 significant digits. The rounding of the text sets the accuracy of the file.
Open item. Write OBJ and Collada numbers with 17 significant digits, so the file reads back to the same 64-bit values.
8 calculations in 1 source file. The code is read from the released app when this page loads. It is the calculation people run, not a copy kept for this page.
Tests None found by the review
How every coordinate and normal component is written in an OBJ file.
text = x rounded to 6 decimals
toFixed(6) rounds the exported value to 1e-6 (1 micrometer for coordinates in meters); the unary plus then drops trailing zeros, and −0 is written as 0.
const f6 = (x) => { const v = +x.toFixed(6); return Object.is(v, -0) ? '0' : String(v); };A vertex of the model as it goes into the OBJ file.
(x, y, z) → M · p → (x, z, −y)
The vertex is multiplied by its accumulated placement matrix in 64-bit and its axes are swapped to Y up, which is exact. It is rounded only when the line is written.
const V = (id) => {
if (!vi.has(id)) { vi.set(id, vBase + vs.length); vs.push(zupToYup(apply(wm, d.verts.get(id)))); }
return vi.get(id);
};The flat normal written for a face.
N = Σ pᵢ × pᵢ₊₁, normalized
The normal is a sum of cross products of the world-space corner positions themselves, so far from the origin the terms are much larger than the result. It is then normalized, rounded to 6 decimals, and shared with any other normal that rounds to the same text.
// flat normal from the transformed polygon (mirror handled by the reversed winding)
const pts = f.loops[0].map((v) => apply(wm, d.verts.get(v)));
let nn = [0, 0, 0];
for (let i = 0; i < pts.length; i++) nn = add(nn, cross(pts[i], pts[(i + 1) % pts.length]));
let flatI = null;
const flat = () => (flatI ??= N(zupToYup(norm(mirror ? mul(nn, -1) : nn))));The lines that carry the numbers.
Every component goes through f6: six decimals.
for (const p of vs) lines.push('v ' + p.map(f6).join(' '));
for (const n of norms) lines.push('vn ' + n.map(f6).join(' '));How every number is written in a Collada file.
text = x rounded to 12 significant digits
toPrecision(12) keeps 12 significant digits; a value below 1e-12 in size, and −0, are written as 0.
const fx = (x) => { let v = +(+x).toPrecision(12); if (Object.is(v, -0) || Math.abs(v) < 1e-12) v = 0; return String(v); };The vertex positions and face normals of each definition in the Collada file.
Positions are the definition's own 64-bit coordinates and normals its Newell normals; each number is written with 12 significant digits.
const src = (sid, arr) => `<source id="${sid}"><float_array id="${sid}-array" count="${arr.length}">${arr.map(fx).join(' ')}</float_array>` +
`<technique_common><accessor source="#${sid}-array" count="${arr.length / 3}" stride="3"><param name="X" type="float"/><param name="Y" type="float"/><param name="Z" type="float"/></accessor></technique_common></source>`;The 4×4 matrix of each placement, written row by row.
The 16 entries are reordered from column-major to row-major and each is written with 12 significant digits, so a rotation's entries are rounded at the twelfth digit.
const rowMajor = (m) => [0, 1, 2, 3].map((r) => [0, 1, 2, 3].map((c) => fx(m[c * 4 + r])).join(' ')).join(' ');Decides whether a face is written as one polygon or as triangles in Collada.
every turn (b − a) × (c − b) ≥ −1e-12 × size²
The face is projected to its 2D frame and each corner's turn is compared with a tolerance scaled by the square of the face's largest extent from its first corner.
/** Is a one-loop face convex (every turn the same way, seen from its front)? */
function convexFace(def, f, n) {
const fr = frame(n);
const ps = f.loops[0].map((v) => to2(fr, def.verts.get(v)));
const k = ps.length;
if (k <= 3) return true;
let size = 0;
for (const p of ps) size = Math.max(size, Math.abs(p[0] - ps[0][0]), Math.abs(p[1] - ps[0][1]));
const tol = -1e-12 * size * size;
for (let i = 0; i < k; i++) {
const a = ps[i], b = ps[(i + 1) % k], c = ps[(i + 2) % k];
if ((b[0] - a[0]) * (c[1] - b[1]) - (b[1] - a[1]) * (c[0] - b[0]) < tol) return false;
}
return true;
}The arithmetic is 64-bit, but the method or the data sets the accuracy: a solver tolerance, a mesh, a step size, a datum shift, a table. The limit is stated where the result is shown.