Quantcast
Channel: Raspberry Pi Forums
Viewing all articles
Browse latest Browse all 5028

SDK • Re: Lua math issue

$
0
0
It is better explained in the RP2040 datasheet, see "2.8.3.2.1. Implementation Details" in the section "2.8. Bootrom", one very pertinent part I reproduce below:
The scientific functions are calculated using internal fixed-point representations so accuracy (as measured in ULP error rather than in absolute terms) is poorer in situations where converting the result back to floating point entails a large normalising shift. This occurs, for example, when calculating the sine of a value near a multiple of pi, the cosine of a value near an odd multiple of pi/2, or the logarithm of a value near 1.
Anyway the function results should be usable for most sane applications, but for these functions it would frequently not be possible to reproduce the numerical results exactly on a normal PC.

Statistics: Posted by katak255 — Mon Jul 22, 2024 12:44 pm



Viewing all articles
Browse latest Browse all 5028

Trending Articles