Resources Numerical accuracy Calculation policy

Gerber and drill file numbers

The coordinates and sizes written to Gerber layers, drill files and the job file, and read back from them.

How the precision is kept

Board coordinates are 64-bit millimeters. The Gerber format chosen (4 integer and 6 decimal digits) stores whole nanometers, so every coordinate is rounded to 1e-6 mm when written. The drill file is coarser: positions to 1e-4 mm and tool diameters to 1e-3 mm. The job file states the board size to 1e-4 mm.

Open item. Write drill tool diameters with the same 4 decimals used to group them.

Calculations under this policy

7 calculations in 1 source file. The code is read from the released app when this page loads. It is the calculation people run, not a copy kept for this page.

gerber.js Shared shell, every app 64-bit 7 calculations Reviewed Nov 5, 2026

Tests bin/test-engines.mjs, tools/pcb-test/test.mjs

Coordinate and aperture format

64-bit

How a position and an aperture size are written in a Gerber file.

X, Y = round(mm × 10⁶); size = 6 decimals

Coordinates are rounded to the nearest nanometer, the resolution of the declared format. v × 1e6 is rounded to 64-bit before Math.round, which matters only for a value within rounding of half a nanometer. Aperture sizes are fixed to 6 decimals with trailing zeros removed.

gerber.js
lines 14 to 16

const um = (v) => Math.round(v * 1e6);
const coord = (x, y) => 'X' + um(x) + 'Y' + um(y);
const f6 = (v) => (+v).toFixed(6).replace(/\.?0+$/, '') || '0';

Pad as a flash or a region

64-bit

A pad, or its grown copy for the solder mask, in the file.

axis-aligned: aperture (w + 2 × grow) by (h + 2 × grow); else a 4-corner region rotated by the pad angle

A pad within 1e-9° of a right angle is flashed with a rectangle or oval aperture, width and height exchanged at 90° and 270°. Other angles are written as a region whose corners are rotated in 64-bit and then rounded to nanometers.

gerber.js
lines 49 to 58

/** Draw a pad (or its grown copy for mask/clear) with the right aperture or a region. grow in mm per side. */
function drawPad(W, pad, grow, fn, attrs = true) {
  const rot = ((Math.round(pad.rot) % 360) + 360) % 360, ortho = Math.abs(pad.rot - Math.round(pad.rot / 90) * 90) < 1e-9, sw = rot === 90 || rot === 270;
  if (attrs) W.setNet(pad.net, pad.ref, pad.num);
  if (pad.shape === 'circle') { W.flash(W.aperture('C', pad.w + 2 * grow, 0, fn), pad.x, pad.y); return; }
  if (ortho && (pad.shape === 'rect' || pad.shape === 'roundrect')) { const w = (sw ? pad.h : pad.w) + 2 * grow, h = (sw ? pad.w : pad.h) + 2 * grow; W.flash(W.aperture('R', w, h, fn), pad.x, pad.y); return; }
  if (ortho && pad.shape === 'oval') { const w = (sw ? pad.h : pad.w) + 2 * grow, h = (sw ? pad.w : pad.h) + 2 * grow; W.flash(W.aperture('O', w, h, fn), pad.x, pad.y); return; }
  const c = Math.cos(pad.rot * Math.PI / 180), s = Math.sin(pad.rot * Math.PI / 180), hw = pad.w / 2 + grow, hh = pad.h / 2 + grow;
  W.region([[-hw, -hh], [hw, -hh], [hw, hh], [-hw, hh]].map(([x, y]) => [pad.x + x * c - y * s, pad.y + x * s + y * c]));
}

Drill sizes and positions

64-bit

The tool table and hole positions of the drill files.

tool size = 3 decimals; X, Y = 4 decimals of a millimeter

Holes are grouped into tools by diameter rounded to 4 decimals, but the tool table prints 3, so two tools can show the same size. Hole positions are rounded to 0.0001 mm, a hundred times coarser than the copper layers.

gerber.js
lines 117 to 130

/** Excellon drill files: plated (vias, plated pad holes) and non-plated. Decimal metric, absolute. */
export function writeDrill(board, opts = {}) {
  const hs = board.drills(), files = {}, counts = { plated: 0, nonplated: 0 };
  for (const plated of [true, false]) {
    const list = hs.filter((h) => h.plated === plated); if (!list.length) continue;
    const sizes = [...new Set(list.map((h) => +h.drill.toFixed(4)))].sort((a, b) => a - b), n = board.layers.length;
    const out = ['M48', '; DRILL file {CadCraft lib-pcb} date ' + (opts.date || '1970-01-01T00:00:00+00:00'), '; FORMAT={-:-/ absolute / metric / decimal}', '; #@! TF.FileFunction,' + (plated ? 'Plated,1,' + n + ',PTH' : 'NonPlated,1,' + n + ',NPTH'), '; #@! TF.FilePolarity,Positive', '; #@! TF.GenerationSoftware,CadCraft,lib-pcb,1.0', 'FMAT,2', 'METRIC'];
    sizes.forEach((s, i) => out.push('T' + (i + 1) + 'C' + s.toFixed(3)));
    out.push('%', 'G90', 'G05');
    sizes.forEach((s, i) => { out.push('T' + (i + 1)); for (const h of list.filter((q) => +q.drill.toFixed(4) === s)) out.push('X' + h.x.toFixed(4) + 'Y' + h.y.toFixed(4)); });
    out.push('M30', ''); files[plated ? 'PTH.drl' : 'NPTH.drl'] = out.join('\n'); counts[plated ? 'plated' : 'nonplated'] = list.length;
  }
  return { files, counts };
}

Board size in the job file

64-bit

The board's width and height stated in the job file.

size = extent of the outline, 4 decimals

The outline's bounding box, each side rounded to 0.0001 mm with toFixed and turned back into a number.

gerber.js
lines 132 to 138

function gerberJob(board, layerFiles, drill, opts, nameBase) {
  const [x0, y0, x1, y1] = bbox(board.outline);
  return { Header: { GenerationSoftware: { Vendor: 'CadCraft', Application: 'lib-pcb', Version: '1.0' }, CreationDate: opts.date || '1970-01-01T00:00:00+00:00' },
    GeneralSpecs: { ProjectId: { Name: board.name, GUID: uuidFor(board.name), Revision: opts.revision || 'A' }, Size: { X: +(x1 - x0).toFixed(4), Y: +(y1 - y0).toFixed(4) }, LayerNumber: board.layers.length, BoardThickness: board.thickness },
    FilesAttributes: [...layerFiles.map((l) => ({ Path: l.name, FileFunction: l.fn, FilePolarity: l.pol })), ...Object.keys(drill.files).map((k) => ({ Path: nameBase + '-' + k, FileFunction: k.startsWith('PTH') ? 'Plated,1,' + board.layers.length + ',PTH' : 'NonPlated,1,' + board.layers.length + ',NPTH', FilePolarity: 'Positive' }))],
    MaterialStackup: board.layers.map((l, i) => ({ Type: 'Copper', Name: l, ...(i < board.layers.length - 1 ? {} : {}) })) };
}

Coordinate scale when reading

64-bit

The factor that turns the file's integers into coordinates.

scale = 10^(−decimals of X)

The scale is Math.pow(10, −d), which is not an exact binary number. Only the X digits of the format statement are used; the Y digits are assumed the same.

gerber.js
lines 157 to 159

if ((m = /^FSLAX(\d)(\d)Y(\d)(\d)$/.exec(t))) { res.fs = { int: +m[1], dec: +m[2] }; scale = Math.pow(10, -m[2]); }
else if (/^FS/.test(t)) problems.push(problem('gerber.read.format', 'Unsupported coordinate format: ' + t + '.'));
else if ((m = /^MO(MM|IN)$/.exec(t))) res.units = m[1];

Coordinates when reading

64-bit

Each X and Y of a Gerber file as a number.

x = integer × scale

One multiplication per coordinate; because the scale is not exact, the product can differ from the decimal value in the last bit. Values stay in the file's unit: the unit is recorded but inch files are not converted to millimeters. A coordinate that is left out keeps the last value.

gerber.js
lines 176 to 178

if (m && (m[1] !== undefined || m[2] !== undefined || m[5])) {
  const nx = m[1] !== undefined ? +m[1] * scale : x, ny = m[2] !== undefined ? +m[2] * scale : y, d = m[5];
  if (d === '2') { if (inRegion && region.length) { /* a region contour restarts */ } x = nx; y = ny; if (inRegion) region = region.length ? [...region] : []; if (inRegion) region.push([x, y]); continue; }

Drill file when reading

64-bit

Tool diameters and hole positions read back from a drill file.

decimal text to the nearest 64-bit number

Decimal text is converted with the unary plus operator, which gives the nearest 64-bit number to each printed value. A file that is not metric gets a warning and its numbers are returned as read.

gerber.js
lines 191 to 203

/** Read an Excellon drill file as written above (decimal metric): {tools: {T1: dia}, holes: [{x, y, dia}], attrs}. */
export function readDrill(text) {
  const tools = {}, holes = [], attrs = [], problems = []; let cur = null, body = false, metric = false;
  for (const raw of text.split(/\r?\n/)) {
    const l = raw.trim(); if (!l) continue;
    if (l.startsWith('; #@!')) { attrs.push(l.slice(5).trim()); continue; } if (l.startsWith(';')) continue;
    if (l === '%') { body = true; continue; } if (l === 'M48') continue; if (l === 'METRIC') { metric = true; continue; }
    let m; if (!body && (m = /^T(\d+)C([\d.]+)/.exec(l))) { tools['T' + m[1]] = +m[2]; continue; }
    if (body) { if ((m = /^T(\d+)$/.exec(l))) { cur = 'T' + m[1]; continue; } if ((m = /^X(-?[\d.]+)Y(-?[\d.]+)$/.exec(l))) { if (!cur || !tools[cur]) problems.push(problem('drill.read.no-tool', 'A hole before any tool.')); holes.push({ x: +m[1], y: +m[2], dia: tools[cur] }); continue; } if (l === 'M30') continue; }
  }
  if (!metric) problems.push(problem('drill.read.units', 'The drill file is not metric.', { severity: 'warning' }));
  return { tools, holes, attrs, problems };
}

In detail

Method: X and Y as Math.round(v × 10⁶); aperture sizes with toFixed(6); drill positions with toFixed(4) and tool sizes with toFixed(3); the reader multiplies integers by 10^(−decimals).
Number types: 64-bit numbers; integers and decimal text in the files.
Tolerances: A pad counts as axis-aligned when its rotation is within 1e-9° of a multiple of 90°.
Limits: Drill holes whose diameters agree to 4 decimals share a tool; diameters that differ in the 4th decimal get separate tools whose header shows the same 3-decimal size. The reader records inch units but does not convert them.

Approach: 64-bit, stated limit

The arithmetic is 64-bit, but the method or the data sets the accuracy: a solver tolerance, a mesh, a step size, a datum shift, a table. The limit is stated where the result is shown.

Every policy that takes this approach