Ranter
Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Comments
-
Voxera113886yIt all depends on if the calculation uses IEEE float/double or datatype designed for accurate decimal.
IEEE is much faster but cannot handle some decimal numbers like 0.1.
In most cases it can conclude what it should be and round it to a sane number but if you do some calculations it no longer fits the requirements for rounding -
@imerljak Some people actually know how to produce proper software. Unlike certain companies...
-
@irene But still at my company this was news for a mathematician who is not too much into writing software.
-
Float is a expression.
It is not a concrete value.
Expression - depending on the precision / mantisse.
...
So yes. This is exactly what you should expect. -
Yep. Floating point math is broken. Don't use it if close enough isn't good enough. This is why I get annoyed at languages that don't have an int type and people who think that's not an issue.
Here's a great video about it: https://youtu.be/PZRI1IfStY0
Related Rants
Apparently, floating point math is broken.
=SUM((2.1 - 2.0) - 0.1)
In PHP and Haskell this also happens
rant
floating point math
web development
math
excel