Resources Numerical accuracy Calculation policy
How a number in an edit is written into a Studio Live message and read back on the other side.
Edits travel as MessagePack, written and read by this file's own code. A whole number inside the safe integer range is written as an integer; every other number is written as an 8-byte 64-bit float. The writer never produces the 4-byte 32-bit float form, so a coordinate arrives with the same 64 bits it left with.
5 calculations in 1 source file. The code is read from the released app when this page loads. It is the calculation people run, not a copy kept for this page.
Tests tools/live-test/test-client.mjs
Decides how a number is written.
A value that is an integer and a safe integer goes to the integer writer; anything else, including fractions, NaN, the infinities and integers beyond 2^53, goes to the 64-bit float writer.
case 'number': return Number.isInteger(v) && Number.isSafeInteger(v) ? this.int(v) : this.float(v);Writes a number as 8 bytes.
Type byte 0xcb and DataView.setFloat64: all 64 bits are sent. There is no 32-bit float writer in the file.
float(v) { this.u8(0xcb); this.need(8); this.view.setFloat64(this.p, v); this.p += 8; }Writes a whole number in the shortest integer form that holds it.
The forms are chosen by range, from one byte up to 64 bits, and each holds the integer exactly. Zero and negative zero both take the first branch and are written as the byte 0.
int(v) {
if (v >= 0) {
if (v < 128) return this.u8(v);
if (v < 0x100) { this.u8(0xcc); return this.u8(v); }
if (v < 0x10000) { this.u8(0xcd); return this.u16(v); }
if (v < 0x100000000) { this.u8(0xce); return this.u32(v); }
return this.big(BigInt(v));
}
if (v >= -32) return this.u8(v & 0xff);
if (v >= -128) { this.u8(0xd0); this.need(1); this.view.setInt8(this.p, v); this.p += 1; return; }
if (v >= -32768) { this.u8(0xd1); this.need(2); this.view.setInt16(this.p, v); this.p += 2; return; }
if (v >= -2147483648) { this.u8(0xd2); this.need(4); this.view.setInt32(this.p, v); this.p += 4; return; }
return this.big(BigInt(v));
}Reads a 4-byte or an 8-byte float from a message.
Type 0xcb is read with getFloat64. Type 0xca is read with getFloat32 and widened; this file's writer never sends it, so a value read that way came from another encoder and carries 32-bit precision.
case 0xca: { this.need(4); const x = v.getFloat32(this.p); this.p += 4; return x; }
case 0xcb: { this.need(8); const x = v.getFloat64(this.p); this.p += 8; return x; }Reads an unsigned or signed 64-bit integer.
The value is read as a BigInt and turned into a number only when it is within ±9007199254740991, where a 64-bit float holds it exactly; otherwise it stays a BigInt.
case 0xcf: { this.need(8); const x = v.getBigUint64(this.p); this.p += 8; return x <= 9007199254740991n ? Number(x) : x; }
case 0xd0: { this.need(1); const x = v.getInt8(this.p); this.p += 1; return x; }
case 0xd1: { this.need(2); const x = v.getInt16(this.p); this.p += 2; return x; }
case 0xd2: { this.need(4); const x = v.getInt32(this.p); this.p += 4; return x; }
case 0xd3: { this.need(8); const x = v.getBigInt64(this.p); this.p += 8; return x >= -9007199254740991n && x <= 9007199254740991n ? Number(x) : x; }Every value is a 64-bit number from input to result. Formulas are closed form or a direct solve, and every comparison uses a stated tolerance.