Resources Numerical accuracy Calculation policy

Half floats in EXR

Converts between 16-bit half floats and 32-bit floats for EXR channels.

How the precision is kept

Every half value converts exactly to a 32-bit float through a 65536 entry Float32Array table. The reverse rounds to the nearest half, ties to even, but it first stores the input in a 32-bit float.

Open item. Round the 64-bit input to half directly instead of through a 32-bit value.

Calculations under this policy

2 calculations in 1 source file. The code is read from the released app when this page loads. It is the calculation people run, not a copy kept for this page.

exr/util.js BimCraft 0.8.1 32-bit in part 2 calculations Reviewed

Tests None found by the review

Half to float

32-bit in part

The value of a 16-bit half float.

v = ± (1 + m ÷ 1024) × 2^(e − 15)

Table lookup; the table is filled in 64-bit as s × m × 2⁻²⁴ or s × (1 + m ÷ 1024) × 2^(e − 15) and stored in 32-bit, which is exact for every half value.

  • exr/util.js
  • lines 39 to 44
  • BimCraft 0.8.1
for (let h = 0; h < 65536; h++) {
  const s = h & 0x8000 ? -1 : 1;
  const e = (h >> 10) & 0x1f;
  const m = h & 0x3ff;
  HALF_TABLE[h] = e === 0 ? s * m * 2 ** -24 : e === 31 ? (m ? NaN : s * Infinity) : s * (1 + m / 1024) * 2 ** (e - 15);
}

Float to half

32-bit in part

The 16-bit half closest to a number.

The input is stored in a Float32Array first, so it is rounded to 32-bit. Then the dropped bits are rounded to nearest even (subnormals use a shift and the same tie rule, a carry from the mantissa moves into the exponent); overflow gives infinity and NaN stays NaN.

  • exr/util.js
  • lines 49 to 75
  • BimCraft 0.8.1
export function floatToHalf(f) {
  F32[0] = f;
  const x = U32[0];
  const sign = (x >>> 16) & 0x8000;
  const e = (x >>> 23) & 0xff;
  let m = x & 0x7fffff;
  if (e === 0xff) return sign | 0x7c00 | (m ? 0x200 | (m >>> 13) : 0);   // Inf, NaN (kept a NaN)
  let he = e - 127 + 15;
  if (he >= 31) return sign | 0x7c00;                                   // overflow → Inf
  if (he <= 0) {                                                        // subnormal half (or zero)
    if (he < -10) return sign;
    m |= 0x800000;
    const shift = 14 - he;
    let hm = m >>> shift;
    const rem = m & ((1 << shift) - 1);
    const half = 1 << (shift - 1);
    if (rem > half || (rem === half && (hm & 1))) hm++;
    return sign | hm;
  }
  let hm = m >>> 13;
  const rem = m & 0x1fff;
  if (rem > 0x1000 || (rem === 0x1000 && (hm & 1))) {
    hm++;
    if (hm === 0x400) { hm = 0; he++; if (he >= 31) return sign | 0x7c00; }
  }
  return sign | (he << 10) | hm;
}

In detail

Method
Table of all 65536 half values; bit rounding of the 32-bit pattern with explicit handling of the subnormal range and of a carry into the exponent.
Number types
Float32Array(1) with a Uint32Array view, Float32Array(65536) table.
Tolerances
A value below 2^-25 in magnitude (he < -10 in the code) becomes zero with its sign; there are no other tolerances.
Limits
A 64-bit input is rounded to 32-bit first, so a value close to a half midpoint can round differently from rounding the 64-bit value directly.

Approach: 32-bit step, flagged

One step stores or exchanges 32-bit values that reach a number a person reads. It is listed here with its open item, and the step is either moved to 64-bit or the app will warn where the number is read.

Every policy that takes this approach