Resources Numerical accuracy

Calculations

Every calculation under our calculation policies: what it works out, its formula and the precision it runs at. Open one to see its code as it is in the released app.

  • 6,917 calculations in 1,248 source files
  • 143 exact
  • 6,286 in 64-bit
  • 335 with a 32-bit step
  • Last reviewed

Calculations

  • Resource utilization

    The share of a shared resource's capacity that was in use.

    utilization = average in use ÷ average capacity

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Vehicle trip times

    How long a vehicle takes to reach the pick station and to carry the load to the drop station.

    to pick = distance ÷ speed + load time; carry = distance ÷ speed + unload time

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Student t factor

    The factor for a 95 % confidence range.

    df ≤ 30: table value; df ≤ 60: 2.0; df ≤ 120: 1.98; above: 1.96

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Replication mean and confidence half-width

    The mean of each result over independent runs and its 95 % range.

    mean = Σx ÷ n; sd = √(Σ(x − mean)² ÷ (n − 1)); half = t × sd ÷ √n

    Flow simulation (discrete events) engine.js Every app
    64-bit
  • Uniform and normal draws

    The random numbers every arrival, service time and route choice is drawn from.

    u = (k + 0.5) ÷ 2³², k a 32-bit integer; z = √(−2 ln u) × cos(2π v)

    Flow simulation (discrete events) rng.js Every app
    64-bit
  • Draw from a distribution

    One value from an exponential, uniform, triangular, normal, lognormal, Erlang, Weibull or empirical distribution.

    exp: −mean × ln u; tri: a + √(u (b − a)(c − a)) or b − √((1 − u)(b − a)(b − c)); weibull: scale × (−ln u)^(1 ÷ shape)

    Flow simulation (discrete events) rng.js Every app
    64-bit
  • Mean of a distribution

    The mean used for load estimates and for routing to the least loaded target.

    tri: (a + m + b) ÷ 3; weibull: scale × Γ(1 + 1 ÷ shape)

    Flow simulation (discrete events) rng.js Every app
    64-bit
  • Gamma function

    Γ(x), used for the mean of a Weibull distribution.

    Lanczos approximation, g = 7, 9 coefficients

    Flow simulation (discrete events) rng.js Every app
    64-bit
  • Thermal voltage

    kT/q, the voltage scale of every diode and transistor junction.

    VT = 0.025852 V

    Circuit solver circuit.js Every app
    64-bit
  • Value with an engineering suffix

    4.7k, 100n, 1meg: a component value read from text.

    value = number × factor of the suffix (t, g, meg, k, m, u, n, p, f); mil = 25.4e-6

    Circuit solver circuit.js Every app
    64-bit
  • Source waveform value

    The value of a DC, sine or pulse source at a time.

    sin: off + amp × e^(−damp (t − delay)) × sin(2π f (t − delay) + phase); pulse: linear edges over tr and tf

    Circuit solver circuit.js Every app
    64-bit
  • Diode current and conductance

    The current through a diode and its slope at a junction voltage.

    i = Is (e^(v ÷ (n VT)) − 1); g = Is e^(v ÷ (n VT)) ÷ (n VT)

    Circuit solver circuit.js Every app
    64-bit
  • Transistor currents

    Collector and base current of a bipolar transistor.

    ic = If − Ir − Ir ÷ βr; ib = If ÷ βf + Ir ÷ βr; If = Is (e^(vbe ÷ VT) − 1)

    Circuit solver circuit.js Every app
    64-bit
  • MOSFET level 1 current

    Drain current of a MOSFET in the triode and saturation regions.

    triode: id = kp (vov vds − vds² ÷ 2)(1 + λ vds); saturation: id = kp vov² (1 + λ vds) ÷ 2

    Circuit solver circuit.js Every app
    64-bit
  • Switch conductance

    The conductance of a voltage-controlled switch.

    ln g = ln(1 ÷ roff) + a (ln(1 ÷ ron) − ln(1 ÷ roff)); a = 1 ÷ (1 + e^(−(v − vt) ÷ vh))

    Circuit solver circuit.js Every app
    64-bit
  • Circuit equations

    The matrix and right side for one Newton step, at DC or at one time step.

    capacitor: geq = C ÷ h (backward Euler) or 2C ÷ h (trapezoid); inductor: r = L ÷ h or 2L ÷ h

    Circuit solver circuit.js Every app
    64-bit
  • Newton iteration

    Repeats the linear solve until the voltages and currents stop changing.

    stop when |Δxᵢ| ≤ 1e-3 × max(|xᵢ new|, |xᵢ old|) + 1e-6 V (or 1e-12 A) for every unknown

    Circuit solver circuit.js Every app
    64-bit
  • Operating point

    The DC voltages and currents of the circuit.

    direct Newton, then gmin stepping, then source stepping

    Circuit solver circuit.js Every app
    64-bit
  • DC sweep

    The circuit solved for each value of a swept source.

    Circuit solver circuit.js Every app
    64-bit
  • AC small-signal response

    Complex node voltages and branch currents at each frequency.

    capacitor: jωC; inductor: −jωL on its branch row; ω = 2π f

    Circuit solver circuit.js Every app
    64-bit
  • Transient

    Voltages and currents over time.

    trapezoidal rule after a first backward-Euler step; output at t = k × tstep

    Circuit solver circuit.js Every app
    64-bit
  • Junction voltage limiting

    Keeps a diode or transistor voltage from jumping too far in one iteration.

    v = v_old + vt ln(1 + (v_new − v_old) ÷ vt) when the step exceeds 2 vt above vcrit

    Circuit solver circuit.js Every app
    64-bit
  • Sweep and frequency lists

    The source values of a DC sweep and the frequencies of an AC analysis.

    dc: start, start + step, …; ac dec or oct: f = fstart × (fstop ÷ fstart)^(k ÷ N); lin: fstart + (fstop − fstart) k ÷ (points − 1)

    Circuit solver circuit.js Every app
    64-bit
  • Magnitude and phase

    The size and angle of a complex result.

    |z| = hypot(re, im); phase = atan2(im, re) × 180 ÷ π

    Circuit solver circuit.js Every app
    64-bit
  • Real linear solve

    Solves the circuit equations of one Newton step.

    LU with partial pivoting, then back substitution

    Circuit solver linalg.js Every app
    64-bit
  • Complex linear solve

    Solves the AC equations and the antenna's impedance matrix.

    LU with partial pivoting on separate real and imaginary arrays

    Circuit solver linalg.js Every app
    64-bit
  • Magnetic and electric constants

    μ0 and ε0 used by every field result.

    μ0 = 4π × 10⁻⁷ H/m; ε0 = 8.8541878128 × 10⁻¹² F/m

    Magnetic and electric field solvers fields.js Every app
    64-bit
  • Field of straight wire segments

    The magnetic flux density at a point from current-carrying segments.

    B = μ0 I ÷ (4π d) × (z_b ÷ √(z_b² + d²) − z_a ÷ √(z_a² + d²))

    Magnetic and electric field solvers fields.js Every app
    64-bit
  • Loop as straight segments

    A circular loop turned into chords for the segment formulas.

    n chords with their ends on the circle, 128 by default

    Magnetic and electric field solvers fields.js Every app
    64-bit
  • Mutual inductance of two loops

    The Neumann integral over two loops given as segments.

    M = μ0 ÷ 4π × Σ Σ (dl₁ · dl₂) ÷ R

    Magnetic and electric field solvers fields.js Every app
    64-bit
  • Complete elliptic integrals

    K(k) and E(k), used for coaxial loops.

    K = π ÷ (2a) at the limit of the arithmetic–geometric mean; E = K (1 − Σ 2ⁿ⁻¹ cₙ²)

    Magnetic and electric field solvers fields.js Every app
    64-bit
  • Mutual inductance of coaxial loops

    Maxwell's formula for two circular loops on one axis.

    M = μ0 √(r₁ r₂) ((2 ÷ k − k) K − (2 ÷ k) E); k² = 4 r₁ r₂ ÷ ((r₁ + r₂)² + d²)

    Magnetic and electric field solvers fields.js Every app
    64-bit
  • 2D finite-element solve

    The potential at every mesh node and its gradient in every triangle.

    ∇·(k ∇u) = −s on linear triangles; element matrix k (bᵢ · bⱼ) A

    Magnetic and electric field solvers fields.js Every app
    64-bit
  • Magnetic field, energy and saturating iron

    Flux density per triangle, stored energy per meter, and the iteration for a B–H curve.

    B = (∂A/∂y, −∂A/∂x); W = Σ ½ ν B² A

    Magnetic and electric field solvers fields.js Every app
    64-bit
  • Electric field and energy

    Field per triangle and stored energy per meter.

    E = −∇V; W = Σ ½ ε E² A

    Magnetic and electric field solvers fields.js Every app
    64-bit
  • Capacitance from energy

    Capacitance per meter from the stored energy.

    C = 2W ÷ ΔV²

    Magnetic and electric field solvers fields.js Every app
    64-bit
  • Force from the Maxwell stress

    The force per meter on a body in the solved field.

    F = Σ ((B · n) B − ½ B² n) × edge length ÷ μ

    Magnetic and electric field solvers fields.js Every app
    64-bit
  • Conjugate-gradient solve

    Solves the finite-element equations.

    stop when ‖r‖₂ ≤ tol × ‖b‖₂

    Magnetic and electric field solvers linalg.js Every app
    64-bit
  • Constants

    The speed of light, μ0, ε0 and the impedance of free space.

    c = 299792458 m/s; μ0 = 4π × 10⁻⁷; ε0 = 1 ÷ (μ0 c²); η0 = μ0 c

    Wire antenna (method of moments) wire.js Every app
    64-bit
  • Gauss–Legendre points

    The sample points and weights of the integrals along segments.

    2, 3 and 4 points on [0, 1]

    Wire antenna (method of moments) wire.js Every app
    64-bit
  • Segments and node merging

    How wires are cut into segments and joined at shared points.

    segments per leg = ⌈L ÷ maxSeg⌉, maxSeg = λ ÷ 20 by default; merge distance = 1e-6 × max(largest coordinate, λ ÷ 1000)

    Wire antenna (method of moments) wire.js Every app
    64-bit
  • Kernel integrals over a segment pair

    The coupling between two segments.

    ∫∫ cos(kR) ÷ (4πR), R = √(|r − r′|² + a²)

    Wire antenna (method of moments) wire.js Every app
    64-bit
  • Impedance matrix

    The matrix that links the currents to the source voltage.

    Z_mn = jωμ (u · u′) ∫∫ f_m f_n G + (1 ÷ jωε) ∫∫ f_m′ f_n′ G

    Wire antenna (method of moments) wire.js Every app
    64-bit
  • Input impedance

    The impedance seen at the feed point.

    Z_in = V ÷ I at the feed

    Wire antenna (method of moments) wire.js Every app
    64-bit
  • Radiation intensity

    Power per solid angle in a direction.

    U = η0 k² ÷ (32π²) × (|N_θ|² + |N_φ|²)

    Wire antenna (method of moments) wire.js Every app
    64-bit
  • Radiated power and directivity

    Total radiated power, the strongest direction and the directivity.

    P = Σ U sin θ Δθ Δφ; D = 4π U_max ÷ P

    Wire antenna (method of moments) wire.js Every app
    64-bit
  • Rotation about an axis

    The rotation matrix for an angle about an axis.

    R = cos θ I + sin θ [a]× + (1 − cos θ) a aᵀ

    Rotations, transforms and small linear algebra (motion) math.js Every app
    64-bit
  • Roll, pitch and yaw from a matrix

    The three angles of a rotation matrix.

    pitch = asin(−R₃₁); roll = atan2(R₃₂, R₃₃); yaw = atan2(R₂₁, R₁₁)

    Rotations, transforms and small linear algebra (motion) math.js Every app
    64-bit
  • Rotation vector from a matrix

    Axis times angle of a rotation, used for orientation error.

    θ = acos((tr R − 1) ÷ 2); v = θ ÷ (2 sin θ) × (R₃₂ − R₂₃, R₁₃ − R₃₁, R₂₁ − R₁₂)

    Rotations, transforms and small linear algebra (motion) math.js Every app
    64-bit
  • Rigid transforms

    Product, inverse and application of a rotation with a translation.

    A·B = (R_A R_B, p_A + R_A p_B); A⁻¹ = (R_Aᵀ, −R_Aᵀ p_A)

    Rotations, transforms and small linear algebra (motion) math.js Every app
    64-bit

A calculation is added only after its source was read. Each statement can be checked in the code shown on its policy's page; the app repositories are public. When a release changes a calculation, its page says so until it has been read again.