Text positions: search hits and the OCR text layer
Works out where a found word sits on the sheet, the cloud drawn round it, and where the words read from a scanned page are placed.
How the precision is kept
These positions are estimates by method. A word inside a text item gets an equal share of the item's width per character, and its height is taken as fixed fractions of the text size. Words read from a scan come from the recognizer's pixel boxes on a picture of about 300 dpi. The arithmetic is 64-bit.
Calculations under this policy
6 calculations in 2 source files. The code is read from the released app when this page loads. It is the calculation people run, not a copy kept for this page.
The box on the sheet round some characters of one piece of the drawing's text.
s = width × i ÷ n; box corners = origin + u·s + v·t, t from −0.22 to 0.9 of the size
Characters are given equal shares of the item's width, so the box of a word inside a longer item is an estimate for proportional fonts. Without a width from pdf.js the width is taken as 0.5 × size per character. The direction vectors are (a, b) and (c, d) of the item's transform, each divided by its Math.hypot length.
textsearch.js
lines 35 to 48
MarkupCraft 0.8.1
/** The box of characters [i0, i1) of one item, in user space. */
function partBox(it, i0, i1) {
const [a, b, c, d, e, f] = it.transform || [1, 0, 0, 1, 0, 0];
const n = Math.max(1, String(it.str).length);
const len = Math.hypot(a, b) || 1, up = Math.hypot(c, d) || 1;
const ux = a / len, uy = b / len, vx = c / up, vy = d / up;
const size = up;
const w = Number.isFinite(it.width) && it.width > 0 ? it.width : n * size * 0.5;
const s0 = (w * i0) / n, s1 = (w * i1) / n;
const lo = -0.22 * size, hi = 0.9 * size; // descenders and cap height around the baseline
const pts = [[s0, lo], [s1, lo], [s1, hi], [s0, hi]].map(([s, t]) => [e + ux * s + vx * t, f + uy * s + vy * t]);
const xs = pts.map((p) => p[0]), ys = pts.map((p) => p[1]);
return [Math.min(...xs), Math.min(...ys), Math.max(...xs), Math.max(...ys)];
}
Cloud round a search hit
64-bit
The box of the cloud markup placed round a found word.
box = hit box grown by pad on each side
Four 64-bit additions or subtractions of the pad, 4 points unless the caller gives another. The stored box inherits the estimate in the hit's box.
The scale at which a scanned page is drawn before its text is read.
s = min(dpi ÷ 72, maxSide ÷ max(w, h), √(maxPixels ÷ (w·h))), at least 0.05
Defaults: 300 dpi, 6000 pixels a side, 36 million pixels. A large sheet is therefore read at less than 300 dpi, which makes each pixel, and so each word position, coarser on the sheet.
ocr.js
lines 50 to 55
MarkupCraft 0.8.1
export function ocrScale(w, h, { dpi = OCR_DPI, maxSide = 6000, maxPixels = 36e6 } = {}) {
let s = dpi / 72;
s = Math.min(s, maxSide / Math.max(w, h, 1));
s = Math.min(s, Math.sqrt(maxPixels / Math.max(1, w * h)));
return Math.max(0.05, s);
}
Matrix that places the text layer
64-bit
The matrix that puts the recognizer's text-only page onto the sheet, written into the saved PDF.
Two 64-bit divisions for the points-to-pixels scale, then one matrix inverse and one product from geom.js. That inverse returns the identity for a determinant of exactly zero. The six numbers are passed on unrounded.
ocr.js
lines 64 to 72
MarkupCraft 0.8.1
/**
* The text-only page (points, y up, as big as the picture at its dpi) → PDF user space of the sheet: points → picture pixels
* (y down), then back through the viewport the picture was drawn with.
*/
export function ocrMatrix(vpT, picW, picH, mediaW, mediaH) {
const kx = picW / mediaW, ky = picH / mediaH;
const toPixels = [kx, 0, 0, -ky, 0, picH];
return mul(inv(vpT), toPixels);
}
Words placed on the sheet
64-bit
The position, direction and size of each word read from a scan, for Find and Select text.
baseline y at x by linear interpolation along the line's baseline; size = height above the baseline ÷ 0.9
Word boxes and baselines are the recognizer's pixel values, taken back through the inverse viewport in 64-bit. The baseline is clamped to lie inside the word's box, at least 1 pixel below its top. The size is at least 0.5 points and the up direction is made square to the baseline.
ocr.js
lines 74 to 104
MarkupCraft 0.8.1
/**
* Words → text items in user space. Each word's baseline comes from its line's baseline (Tesseract gives it in pixels),
* its size so that textsearch's box (−0.22 … 0.9 of the size round the baseline) reaches the word's top.
* `minConf`: words read with less confidence (0–100) are left out (none by default: search finds what Tesseract found).
*/
export function ocrItems(blocks, vpT, { minConf = 0 } = {}) {
const back = inv(vpT);
const items = [];
for (const b of blocks || []) for (const par of b.paragraphs || []) for (const line of par.lines || []) {
const words = (line.words || []).filter((w) => /\S/.test(w.text || '') && (w.confidence ?? 100) >= minConf);
const bl = line.baseline || null;
words.forEach((w, i) => {
const { x0, y0, x1, y1 } = w.bbox;
const baseAt = (x) => (bl && bl.x1 !== bl.x0 ? bl.y0 + (bl.y1 - bl.y0) * (x - bl.x0) / (bl.x1 - bl.x0) : y1);
const by0 = Math.min(y1, Math.max(y0 + 1, baseAt(x0))), by1 = Math.min(y1, Math.max(y0 + 1, baseAt(x1)));
const p0 = apply(back, [x0, by0]), p1 = apply(back, [x1, by1]);
const len = Math.hypot(p1[0] - p0[0], p1[1] - p0[1]) || 1;
const ux = (p1[0] - p0[0]) / len, uy = (p1[1] - p0[1]) / len;
// "up" in user space: the picture's up (−y) taken through the viewport, kept square to the baseline
const t = apply(back, [x0, by0 - 10]);
let vx = t[0] - p0[0], vy = t[1] - p0[1];
const side = ux * vy - uy * vx >= 0 ? 1 : -1;
vx = -uy * side; vy = ux * side;
const topPx = Math.max(1, by0 - y0);
const top = apply(back, [x0, by0 - topPx]);
const size = Math.max(0.5, Math.hypot(top[0] - p0[0], top[1] - p0[1]) / 0.9);
items.push({ str: w.text, transform: [ux * size, uy * size, vx * size, vy * size, p0[0], p0[1]], width: len, height: size, hasEOL: i === words.length - 1, ocr: true, conf: w.confidence ?? null });
});
}
return items;
}
Word count and mean confidence
64-bit
The number of words read on a page and their mean confidence, for the status line.
conf = round(Σ confidence ÷ n)
A plain mean over the words that carry a finite confidence, rounded to a whole number. The word count is the number of items.
ocr.js
lines 106 to 111
MarkupCraft 0.8.1
/** Word count and mean confidence of a page's result (for the status line). */
export function ocrSummary(items) {
const n = items.length;
const conf = items.filter((i) => Number.isFinite(i.conf));
return { words: n, conf: conf.length ? Math.round(conf.reduce((s, i) => s + i.conf, 0) / conf.length) : null };
}
In detail
Method
Search hit: box along the text's own direction from the item's transform, width shared equally among characters, from 0.22 of the size below the baseline to 0.9 above it. OCR: picture scale min(300 ÷ 72, 6000 ÷ longest side, √(36e6 ÷ area)); placement matrix = inverse viewport × (points to pixels); each word's baseline interpolated along its line's baseline.
Number types
64-bit numbers. Word boxes arrive as pixel coordinates from the recognizer.
Tolerances
No comparison tolerances. Text size from OCR is at least 0.5 points; the picture scale is at least 0.05.
Limits
The box of part of a text item is exact only when all its characters are equally wide. At 300 dpi one pixel of the scan is 0.24 points on the sheet.
Approach: 64-bit, stated limit
The arithmetic is 64-bit, but the method or the data sets the accuracy: a solver tolerance, a mesh, a step size, a datum shift, a table. The limit is stated where the result is shown.