Adjusts a survey network or a leveling run by least squares, closes a traverse by the compass rule, and reports residuals, the variance factor and error ellipses.
How the precision is kept
All arithmetic is 64-bit. The plan adjustment is a Gauss-Newton iteration that stops on an absolute correction of 1e-9 or after 20 passes, and it forms and inverts the normal matrix in full each pass. The statistical limits come from approximations with constants written to six or seven digits. The CSV export rounds every value to 4 decimals.
Open item. Write the adjusted coordinates and the ellipse azimuth to the CSV at full 64-bit precision instead of 4 decimals.
Calculations under this policy
10 calculations in 1 source file. The code is read from the released app when this page loads. It is the calculation people run, not a copy kept for this page.
The inverse of the normal matrix, used for the solution and for the cofactors behind the error ellipses.
[N | I] → [I | N⁻¹] by row operations
Gauss-Jordan elimination with partial pivoting on 64-bit rows; the whole inverse is formed. A pivot smaller than 1e-14 in absolute value is treated as singular, and that threshold is not scaled to the size of the matrix entries.
adjust.js
lines 30 to 41
CivilCraft 0.8.1
/** Solve A x = b (dense, n small) by Gauss–Jordan with partial pivoting; returns the inverse when `inverse`. */
export function invert(M) {
const n = M.length, a = M.map((r, i) => [...r, ...Array.from({ length: n }, (_, j) => (i === j ? 1 : 0))]);
for (let c = 0; c < n; c++) {
let p = c; for (let r = c + 1; r < n; r++) if (Math.abs(a[r][c]) > Math.abs(a[p][c])) p = r;
if (Math.abs(a[p][c]) < 1e-14) return null;
[a[c], a[p]] = [a[p], a[c]];
const d = a[c][c]; for (let j = 0; j < 2 * n; j++) a[c][j] /= d;
for (let r = 0; r < n; r++) if (r !== c) { const f = a[r][c]; if (f) for (let j = 0; j < 2 * n; j++) a[r][j] -= f * a[c][j]; }
}
return a.map((r) => r.slice(n));
}
Observation equations
64-bit
For each distance, direction, angle, azimuth, vector or control observation: observed minus computed, and its partial derivatives.
Distances use hypot and azimuths atan2; every angular difference is wrapped to (−π, π] before it is used. The partial derivatives divide by the distance and by its square with no guard, so two points at the same place divide by zero.
Builds the weighted normal equations, solves for the corrections and applies them until they are small.
N = AᵀPA; u = AᵀPl; Δx = N⁻¹u; P = 1 ÷ σ²
The normal matrix is summed term by term in observation order and inverted in full each pass. The loop stops when the largest correction is below 1e-9 (absolute: plan units for coordinates, radians for orientations) or after 20 passes; a run that reaches 20 is reported as not converged.
adjust.js
lines 103 to 114
CivilCraft 0.8.1
for (; it < iterations; it++) {
E = build();
if (E.length < nu) { problems.push({ code: 'adj.underdetermined', observations: E.length, unknowns: nu }); break; }
const N = Array.from({ length: nu }, () => new Array(nu).fill(0)), u = new Array(nu).fill(0);
for (const e of E) { for (const [i, a] of e.rows) { u[i] += a * e.w * e.l; for (const [j, b] of e.rows) N[i][j] += a * e.w * b; } }
Ninv = invert(N);
if (!Ninv) { problems.push({ code: 'adj.singular' }); break; }
const dx = Ninv.map((row) => row.reduce((s, v, j) => s + v * u[j], 0));
free.forEach((id, i) => { pts[id].x += dx[2 * i]; pts[id].y += dx[2 * i + 1]; });
sets.forEach((s, i) => { s.z = wrap(s.z + dx[nP + i]); });
if (Math.max(...dx.map(Math.abs)) < tol) { converged = true; it++; break; }
}
Variance factor, redundancy and standardized residuals
64-bit
The a posteriori standard deviation of unit weight, each observation's redundancy number and its standardized residual.
σ̂₀ = √(vᵀPv ÷ dof); r = 1 − w·aᵀQa; w-test = v ÷ (σ·√r)
vᵀPv is one sum in observation order. The redundancy number is 1 minus a product, so a value near zero has lost digits; at 1e-6 or below the standardized residual is set to 0 instead of dividing by its square root.
adjust.js
lines 118 to 125
CivilCraft 0.8.1
const vtpv = E.reduce((s, e) => s + e.w * e.l * e.l, 0);
const sigma0 = dof > 0 ? Math.sqrt(vtpv / dof) : NaN;
// redundancy numbers r_i = 1 − w_i·a_iᵀ Qxx a_i
const res = E.map((e) => {
let q = 0; for (const [i, a] of e.rows) for (const [j, b] of e.rows) q += a * Ninv[i][j] * b;
const r = 1 - e.w * q, v = -e.l; // residual = adjusted − observed; what remains of (observed − computed) after the last step, with the sign turned
return { k: e.k, sd: e.sd, v, r, w: r > 1e-6 ? v / (e.sd * Math.sqrt(r)) : 0 };
});
Error ellipses
64-bit
The semi-axes and azimuth of each free point's confidence ellipse, and its standard deviations in x and y.
Closed-form eigenvalues of the 2×2 cofactor block. For a thin ellipse the smaller eigenvalue is a difference of nearly equal numbers; it is clamped at zero before the square root. The scale factor is the exact two-degree chi-square factor, 2.4477 at 95 %.
The lower and upper limits the variance test compares vᵀPv with.
χ²(p, k) ≈ k·(1 − 2/(9k) + z·√(2/(9k)))³
Three or more degrees of freedom use the Wilson-Hilferty approximation with z = ±1.959964, written to six decimals. One degree of freedom returns the table values 0.000982 and 5.0239; two use −2 ln(1 − p). A probability other than 0.025 or 0.975 takes z = 0.
adjust.js
lines 45 to 50
CivilCraft 0.8.1
export function chi2Quantile(p, k) {
if (k === 1) return { 0.025: 0.000982, 0.975: 5.0239 }[p] ?? NaN;
if (k === 2) return -2 * Math.log(1 - p);
const z = Z[p] ?? 0, t = 1 - 2 / (9 * k) + z * Math.sqrt(2 / (9 * k));
return k * t * t * t;
}
Critical value for the residual test
64-bit
The two-sided normal critical value for the chosen significance level.
t = √(−2 ln(α/2)); z ≈ t − (c₀ + c₁t + c₂t²) ÷ (1 + d₁t + d₂t² + d₃t³)
A rational approximation with coefficients written to six or seven digits; the comment cites Abramowitz and Stegun 26.2.23. The result is an approximation of the normal quantile, not a 15-digit value.
adjust.js
lines 144 to 148
CivilCraft 0.8.1
/** Two-sided standard-normal critical value for α (0.001 → 3.29; 0.05 → 1.96): rational approximation (Abramowitz and Stegun 26.2.23). */
export function zCrit(alpha) {
const p = alpha / 2, t = Math.sqrt(-2 * Math.log(p));
return t - (2.515517 + 0.802853 * t + 0.010328 * t * t) / (1 + 1.432788 * t + 0.189269 * t * t + 0.001308 * t * t * t);
}
Leveling network
64-bit
Adjusted heights of the free points from observed height differences, with residuals and standard deviations.
weight = 1 ÷ σ², or 1 ÷ length; h = N⁻¹u
The problem is linear, so there is one solve with the explicit inverse and no iteration. Known heights are added to or subtracted from the observation first; residuals are recomputed from the adjusted heights.
adjust.js
lines 150 to 167
CivilCraft 0.8.1
/** Levelling network: heights of the free points from height differences, weights 1/length (or 1/sd²). Fixed points have `fixed: true`. */
export function adjustLevels(heights0, obs) {
const ids = Object.keys(heights0), free = ids.filter((i) => !heights0[i].fixed), idx = new Map(free.map((id, i) => [id, i]));
const n = free.length, N = Array.from({ length: n }, () => new Array(n).fill(0)), u = new Array(n).fill(0), rows = [];
for (const o of obs) {
const w = o.sd ? 1 / (o.sd * o.sd) : 1 / (o.length || 1), a = idx.get(o.from), b = idx.get(o.to);
let l = o.dh; if (a === undefined) l += heights0[o.from].h; if (b === undefined) l -= heights0[o.to].h; // dh = h_to − h_from
const r = []; if (b !== undefined) r.push([b, 1]); if (a !== undefined) r.push([a, -1]);
for (const [i, x] of r) { u[i] += x * w * l; for (const [j, y] of r) N[i][j] += x * w * y; }
rows.push({ o, w, l, r });
}
const Q = invert(N); if (!Q) return { ok: false, problems: [{ code: 'adj.singular' }] };
const h = Q.map((row) => row.reduce((s, v, j) => s + v * u[j], 0)), out = {};
for (const id of ids) out[id] = idx.has(id) ? { ...heights0[id], h: h[idx.get(id)] } : { ...heights0[id] };
const vv = rows.map((x) => { const calc = (x.r.find((q) => q[1] === 1)?.[0] !== undefined ? h[x.r.find((q) => q[1] === 1)[0]] : 0) - (x.r.find((q) => q[1] === -1)?.[0] !== undefined ? h[x.r.find((q) => q[1] === -1)[0]] : 0); return calc - x.l; });
const dof = obs.length - n, vtpv = rows.reduce((s, x, i) => s + x.w * vv[i] * vv[i], 0);
return { ok: true, heights: out, residuals: vv, dof, sigma0: dof > 0 ? Math.sqrt(vtpv / dof) : NaN, sd: Object.fromEntries(free.map((id, i) => [id, Math.sqrt(Q[i][i]) * (dof > 0 ? Math.sqrt(vtpv / dof) : 1)])) };
}
Traverse by the compass rule
64-bit
Traverse coordinates, the misclosure, and each point corrected in proportion to the distance traveled.
x += d·sin(az), y += d·cos(az); correction = misclosure × (length so far ÷ perimeter); precision = perimeter ÷ misclosure
Coordinates accumulate leg by leg, so rounding adds up along the traverse. The misclosure is the difference of two full-size coordinates. The precision ratio is Infinity when the misclosure is exactly zero.
adjust.js
lines 183 to 190
CivilCraft 0.8.1
/** Closed or connecting traverse by the compass (Bowditch) rule: each coordinate difference is corrected in proportion to the leg's length. */
export function bowditch(start, legs, { end = null } = {}) {
const raw = [{ x: start.x, y: start.y }]; let perimeter = 0;
for (const l of legs) { const p = raw[raw.length - 1]; raw.push({ x: p.x + l.dist * Math.sin(l.az), y: p.y + l.dist * Math.cos(l.az) }); perimeter += l.dist; }
const last = raw[raw.length - 1], target = end || start, dx = target.x - last.x, dy = target.y - last.y, len = Math.hypot(dx, dy);
let acc = 0; const points = raw.map((p, i) => { if (i > 0) acc += legs[i - 1].dist; const f = perimeter > 0 ? acc / perimeter : 0; return { x: p.x + dx * f, y: p.y + dy * f }; });
return { points, raw, misclosure: { dx: -dx, dy: -dy, len }, perimeter, precision: len > 0 ? perimeter / len : Infinity };
}
Adjusted points as CSV
64-bit
The export of adjusted coordinates and ellipse values, one point per line.
value written = round(v × 10⁴) ÷ 10⁴
Every exported value is rounded to 4 decimals before it is written: coordinates and ellipse axes to 0.0001 of the plan unit, and the ellipse azimuth to 0.0001 radian, which is about 21 seconds of arc.
adjust.js
lines 192 to 196
CivilCraft 0.8.1
export function adjustedCsv(r) {
const rows = ['id,x,y,semi-major,semi-minor,azimuth']; const f = (v) => (Number.isFinite(v) ? String(Math.round(v * 1e4) / 1e4) : '');
for (const [id, p] of Object.entries(r.points || {})) { const e = r.ellipses?.[id]; rows.push([String(id).replace(/[",\n]/g, ' '), f(p.x), f(p.y), f(e?.semiMajor), f(e?.semiMinor), f(e?.azimuth)].join(',')); }
return rows.join('\n') + '\n';
}
In detail
Method
Parametric (observation equation) least squares by Gauss-Newton, weighted by 1 ÷ σ². The file header cites Ghilani and Wolf, Adjustment Computations, 6th ed., ch. 11 to 16, and Baarda 1968 for the standardized residual test. Traverses use the compass (Bowditch) rule.
Number types
64-bit numbers in plain arrays. No typed arrays.
Tolerances
Convergence: largest correction below 1e-9, absolute, in plan units for coordinates and radians for orientations; at most 20 passes. Singular matrix: pivot below 1e-14, absolute. Standardized residual set to 0 when the redundancy number is 1e-6 or less. An observation is flagged when |w| exceeds the critical value and its redundancy number is above 0.05. A weighted control point with one standard deviation missing gets 1e-6 for it.
Limits
The normal matrix is inverted explicitly, so the solution carries the square of the condition number of the design matrix. Chi-square limits above two degrees of freedom are the Wilson-Hilferty approximation. No correction for earth curvature or map scale is applied here.
Approach: 64-bit, stated limit
The arithmetic is 64-bit, but the method or the data sets the accuracy: a solver tolerance, a mesh, a step size, a datum shift, a table. The limit is stated where the result is shown.