Machine epsilon of the HP Prime

Machine epsilon of the HP Prime

+- HP Forums (https://www.hpmuseum.org/forum)
+-- Forum: HP Software Libraries (/forum-10.html)
+--- Forum: HP Prime Software Library (/forum-15.html)
+--- Thread: Machine epsilon of the HP Prime (/thread-18835.html)

Machine epsilon of the HP Prime - cdeaglejr - 09-17-2022 11:28 AM

// program mach_eps HP Prime

// September 17, 2022

// machine epsilon of the HP Prime

// From Wikipedia;

// "Machine epsilon or machine precision is an upper bound
// on the relative approximation error due to rounding in
// floating point arithmetic. This value characterizes
// computer arithmetic in the field of numerical analysis,
// and by extension in the subject of computational science."

/////////////////////////////////////////////////////////////

export mach_eps()

begin

local epsmach;

local one := 1.0;

local two := 2.0;

local u := 1.0;

repeat

u := u / two;

until ((one + u) = one);

epsmach := u;

print();

print("machine epsilon of the HP Prime");

print("\neps = " + epsmach);

print("\n1.0 + eps = " + (1.0 + epsmach));

end;

RE: Machine epsilon of the HP Prime - Albert Chan - 09-20-2022 06:24 AM

For HP Prime, formal definition (maximum relative error) and variant definition (ulp of 1.) are the same.
CAS round number to 53 bits double (round-to-nearest), then chop to 48 bits,

u = maximum relative error = (1 ULP) / (2^47 ULP) = 2^-47

1 = 2^47 ULP, unaffected by rounding-mode.
This probably made variant defintion, ε = ulp of 1., more popular.