FAQ Database Discussion Community

## Decimal numbers not adding correctly in AS3 [duplicate]

actionscript-3,floating-point
This question already has an answer here: Is floating point math broken? 18 answers In my program, I have a score multiplier variable of type 'Number' When I try to add 0.1 to it, I have a problem. Here is the code: scoreMultiplier += 0.1; trace(scoreMultiplier); scoreMultiplier is originally...

## Converting floating point numbers to integers, rounding to 2 decimals in JavaScript

javascript,floating-point,int
What's the best way to perform the following conversions in JavaScript? I have currencies stored as floats that I want rounded and converted to integers. 1501.0099999999999909 -> 150101 12.00000000000001 -> 1200...

## Problems with floating-point additions. Ignoring some small values

math,cuda,floating-point
I'm looking up a book about CUDA. On the chapter which explains the floating points of CUDA, I found something odd. The book says that (1.00 * 1) + (1.00 * 1) + (1.00 * 0.01) + (1.00* 0.01) = 10. All the numbers are binaries. 0.01 refers to decimal...

## Multliplication of std_logic_vector with Floating Point

floating-point,vhdl,fixed-point
I have 32 bit std_logic_vector signal and want to multiply it by floating point . e.g signal Input : std_logic_vector (31 downto 0 ); signal number = 0.2 ; signal Output: std_logic_vector (31 downto 0 ); Output <= 0.2 * Input ; What can be the best solution to do...

## Float value giving extremely large answer (C)

c,floating-point
I'm an extremely new user of C++ and I would like to know why my calculations are producing extremely absurd results. This is the part of code I am having problems with. printf("Please enter the length of side:\n"); scanf("%.f", &lengthCube); volume=lengthCube*lengthCube*lengthCube; printf("The volume of this cube is %.f", volume); volume...

## Rounding price to the nearest multiple of 5cent in java

java,floating-point,number-rounding
I'm new to java programming. I would like to round up a price to the nearest 2 decimal places. E.g. 38.82 into 38.80 38.87 into 38.90 38.85 stays the same. I did the E.g. 1 and E.g. 2 but it comes out only 1 decimal place. E.g. 38.82 to 38.8...

## How can I get full-ranged random float values?

java,random,floating-point
I found that Random#nextFloat returns a value between 0.0 and 1.0. How can I get a random float value such as -72.0F or 126.232F? I currently doing like this. float randomFloat() { final ThreadLocalRandom random = ThreadLocalRandom.current(); float value = random.nextFloat() * Float.MAX_VALUE; if (random.nextBoolean()) { value = 0 -...

## Java: double rounding algorithm

java,floating-point,rounding
I got curious about a rounding algorithm, because in CS we had to emulate an HP35 without using the Math library. We didn't include a rounding algorithm in our final build, but I wanted to do it anyway. public class Round { public static void main(String[] args) { /* *...

## precision of comparing double values with EPSILON in C

c,floating-point,double,epsilon
Doing function that takes 2 arrays (column1 and column2) from struct CSV D and plots the graph from it. Idea is to find max, min values of each array, then break range between min−EPSILON and max+EPSILON in to 600 equal regions, where EPSILON = 10^(−6) Problem is that function does...

## Does this function compute the natural logarithm of a number with an exact precision of 3 digits?

c++,algorithm,floating-point,logarithm
I have the following algorithm in my textbook that should compute the natural log of a number with an exact precision of 3 digits. #include <iostream> #include <cmath> double logN(double a, double li, double ls) { if(a == 1) return 0; else if(fabs(li - ls) < 0.0001) return (li +...

## bash + sed: trim trailing zeroes off a floating point number?

regex,bash,sed,floating-point
I'm trying to get trim off trailing decimal zeroes off a floating point number, but no luck so far. echo "3/2" | bc -l | sed s/0\{1,\}\\$// 1.50000000000000000000 I was hoping to get 1.5 but somehow the trailing zeroes are not truncated. If instead of 0\{1,\} I explicitly write 0...

## Swift: how to compare the first n digits past the decimal place for CGFloat?

swift,floating-point
How can you compare if A and B, both CGFloats, are equal up to 5 digits past the decimal place? This is necessary because of this issue.

## Python: Replace “wrong” floats by integers inside string

python,string,replace,floating-point,integer
I'm trying to replace all useless floats in a string (1.0, 2.0 etc.) by integers. So I'd turn a string like "15.0+abc-3" to "15+abc-3". Do you know a way to do that? I hope you understood my idea. If you didn't feel free to ask....

## How to make the value of the decimal only show in numerals?

c,floating-point,decimal
Im trying to make my c program run any value that you put in will come out as words for example if I wanted to put in the value 1234.56 it should come out as "One Thousand Two Hundred Thirty Four Dollars ... and 56 Cents" Notice how the cents...

## Can the standard math functions handle infinity correctly?

c++,c,math,floating-point
int main() { double inf = INFINITY; double pi = acos(-1.0); printf("[1]: %f %f\n", atan(inf) / pi, atan(-inf) / pi); printf("[2]: %f %f\n", tan(inf) / pi, tan(-inf) / pi); return 0; } outputs [1]: 0.500000 -0.500000 [2]: -nan -nan Is such behaviour defined by the standard? Is [2] undefined behaviour?...

## Write float array into a binary file c#

c#,matlab,file,binary,floating-point
I have a float array : float[] samples32array I need to convert it into a binary file so I can read it in matlab. Is there any way to do that?...

## Is it always safe to negate a floating point number

c++,c,floating-point
Consider: double f = foo(); double g = -f; where foo() can return anything that be assigned to f. is double g = -f; safe in C and C++? For IEEE 754 types it obviously is but C and C++ do not restrict floating point implementation to that (unlike Java)....

## How to change a float64 number to uint64 in a right way?

go,floating-point,type-conversion,floating-point-conversion
package main func main() { var n float64 = 6161047830682206209 println(uint64(n)) } The output will be: 6161047830682206208 It looks like that when float64 change to uint64, the fraction is discarded....

## Submit Decimal/Float in URL Request with Swift

swift,url,post,floating-point,decimal
How can a value such as "1.5" be passed in a a URL POST request using swift? For example: let number = "1.5" let numberValue = number.stringByAddingPercentEscapesUsingEncoding(NSUTF8StringEncoding) let server:String = "www.someserver.com" let phpFile:String = "/php/SomePHPScript.php" let baseURL = NSURL(string: "http://\(server)\(phpFile)") let url = NSURL(string: "?value=\(numberValue)", relativeToURL: baseURL) let cachePolicy =...

## Trouble returning decimal after input from user

python,powershell,floating-point,decimal
I'm working on a number of different ways to use variables in functions, but there is one way I just can't figure out after about an hour of searching the web. I just want the user to be able to enter a decimal number like "23.45" and the program print...

## Translate this float value into english

c++,visual-studio,floating-point
So I was debugging in VS and I found this to be the value of a float variable -1.#INF0000 What is it? negative infinity? If not then what's the INF mean?...

## Can someone help me explain this code that is converting decimal fractions into a binary?

python,while-loop,binary,floating-point,decimal
Can someone help me explain this code that is converting decimal fractions into a binary? Convert the decimal fractions into a binary form: x = float(raw_input('Enter a decimal number between 0 and 1: ')) p = 0 while ((2**p)*x)%1 != 0: print('Remainder = ' + str((2**p)*x - int((2**p)*x))) p +=...

## Fixed number rounding down in JavaScript

javascript,floating-point,rounding,floor
Problem I get numbers from 1 to 5 including all possible floating point numbers in between. The output must contain two digits after comma and in case of after-comma digits they need to be rounded down (floor). Example input and output: 1 -> 1.00 4.3 -> 4.30 1.1000 -> 1.10...

## Understanding DBL_MAX

c,floating-point,double,double-precision
I just read about the IEEE 754 standard in order to understand how single-precision and double-precision floating points are implemented. So I wrote this to check my understanding: #include <stdio.h> #include <float.h> int main() { double foo = 9007199254740992; // 2^53 double bar = 9007199254740993; // 2^53 + 1 printf("%d\n\n",...

## SIGFPE with gcc and unsigned integers

c,gcc,floating-point,c11,sigfpe
I'm getting this weird behaviour from an executable compiled with different versions of gcc, all emit the SIGFPE signal and the best part is that I have no floating point of any kind in my code; if someone could shed some light on this ... I literally don't know where...

## Convert char to float in c

c,floating-point,char,type-conversion
I need to convert char to float. I know we can do this with the help of atof() function. But I dont want to create another variable to hold the float. I want the converted float to go in the same variable. Like this operand = atof(operand) Here operand is...

## Floating point precision issue in PHP and MySQL for storing occasional decimal values

php,mysql,floating-point,floating-point-precision,decimal-point
I want to store occasional decimal values in my MySQL database and display them in my PHP application. Let me explain, what do I mean by occasional decimal values. The numbers are whole numbers at most of the time like an integer. For example, 160 or 170 etc.. But sometimes...

## significant decimal digits of binary32 and binary64

floating-point,double,ieee-754,floating-point-conversion,significant-digits
Accourding to Wikipedia the binary32 format has from 6 to 9 significant decimal digits precision and 64 format has from 15 to 17. I found that these significant decimal digits have been calculated using the Mantissa but i didn't get it how can one calculate it? Any Idea ? Mantissa...

## *ignore* Convert the current time to a float value

android,time,floating-point
I tried converting the current time to a Float value, but the value that is returned isn't a Float My code: float currentTime = Calendar.getInstance().getTime().getHours() + (Calendar.getInstance().getTime().getMinutes()/60); I am not getting a Float value. What am I doing wrong? Thanks in advance! EDIT: This is the code that is working...

## Bigfloat - Error in the precision

python,python-2.7,floating-point,bigfloat
I'm trying to use Bigfloat library in python 2.7. from bigfloat import * f1 = Context(precision=2000) with precision(2000): f1 = 1e-19*1e-19*9e9/((1-1e-18)*(1-1e-18))-1e-19*1e-19*9e9 with precision(100): f2 = 1.6e-27*1.6e-27*6.6e-11/(1e-18*1e-18) print BigFloat(f1) print f2 Python gives me f1=0, but it is not true. I tested it with g++ and the result is 1.75e-46. Is...

## Convert float number to int or string

php,string,floating-point,int
I am retrieving a Facebook ID and I am getting a float: float(1.1262850591603E+16) How could I convert it to int (or string) to use it here? http://graph.facebook.com/1.1262850591603E+16/picture I tried intval function, but it returns a wrong result: <?php intval(1.1262850591603E+16); // returns -1062487752 ?> Thank you!...

## Using float and double for calculation, giving different results [duplicate]

java,floating-point,double
This question already has an answer here: Is floating point math broken? 18 answers Why not use Double or Float to represent currency? 10 answers I am working on a project where we are calculating prices of items with 8 places of decimal points. But sometime calculation results are...

## No really, when does floating point promotion actually happen?

c++,floating-point,promotions
From this other QUESTION they talk about how Bjarne Stroustrup said that just as integral data-types narrower than an int(e.g. short) are promoted to an int, floats are promoted to a double. However, unlike widening of integrals narrower than an int, floating point promotion does not happen in the same...

## Numerical error when converting array to list

python,arrays,list,numpy,floating-point
I'm doing quite a bit of scientific numerical integration in Python, using Numpy and ode. I use several arrays, and I wanted to turn a 1d array into a list for exporting and easier manipulation. Since then I've found easier and more pythonic methods without resorting to lists, but before...

## Converting float to uint64 and uint32 behaves strangely

c++,floating-point,type-conversion
When I convert a 32bit float to a 64bit unsigned integer in C++, everything works as expected. Overflows cause the FE_OVERFLOW flag to be set (cfenv) and return the value 0. std::feclearexcept(FE_ALL_EXCEPT); float a = ...; uint64_t b = a; std::fexcept_t flags; std::fegetexceptflag(&flags, FE_ALL_EXCEPT); But when I convert a 32bit...

## Does OCaml have C-like round() and trunc() functions?

floating-point,ocaml
OCaml's standard library includes several floating-point functions equivalent to C ones, such as mod_float for C's fmod(), the exponentiation operator ** for C's pow(), and other functions such as ceil, log, etc. But does it also include equivalents for round() and trunc()? There is truncate/int_of_float, but their type is float...

## Typecasting int pointer to float pointer

c,pointers,floating-point,int,type-conversion
I'm trying to do the following int a[8]={1,2,3,4,5,6,7,8}; printf("%f\n", *(float *)a); printf("%f\n", *((float *)a+1)); printf("%f\n", *((float *)a+2)); printf("%f\n", *((float *)a+3)); printf("%f\n", *((float *)a+4)); printf("%f\n", *((float *)a+5)); printf("%f\n", *((float *)a+6)); printf("%f\n", *((float *)a+7)); I get 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 The reason why I'm trying to print the...

## Floating-point computation without syscall?

linux-kernel,floating-point,syscall
I am reading Robert Love's book on Linux kernel (which says no FP computation inside kernel). And I am wondering how floating point computation is done in user space. For instance, does 3.14 + 5.26 (in C) invoke any syscall to do the job?...

## For what reason printf does not correctly display the float?

c,casting,floating-point,printf
I hoped that in all cases showed 2.33, however, for what reason only in the second case this happened? printf("Without cast: %0.2f\n", 7 / 3); // Whitout cast: 0.00 printf("With cast: %0.2f\n", (float) 7 / 3); // With cast: 2.33 float x = 7 / 3; printf("With var: %0.2f\n", x);...

## fpscr register is not updated when enabling floating point exceptions on arm7, SIGFPE not generated

debugging,android-ndk,floating-point,arm
fpscr register is not updated and SIGFPE is not generated. This was tested on an NVidia Shield Tablet and a 1st gen Nexus 7. feenableexcept(FE_DIVBYZERO | FE_INVALID | FE_OVERFLOW); The implementation calls code, which eventually executes this assembly: asm _volatile__("vmsr fpscr,%0" : :"ri" (fpscr)); ; disassembly follows ldr r3, [r11,...

## Float comparisons failing without any obvious reason (32-bit X86 on Linux)

c++,linux,floating-point,vectorization
I have stumbled upon an interesting case of comparing (==, !=) float types. I encountered this problem while porting my own software from windows to linux. It's a bit of a bummer. The relevant code is the following: template<class T> class PCMVector2 { public: T x, y; public: bool operator...

## Do floats, doubles, and long doubles have a guaranteed minimum precision?

c++,floating-point,language-lawyer,floating-point-precision,minimum
From my previous question "Is floating point precision mutable or invariant?" I received a response which said, C provides DBL_DIG, DBL_DECIMAL_DIG, and their float and long double counterparts. DBL_DIG indicates the minimum relative decimal precision. DBL_DECIMAL_DIG can be thought of as the maximum relative decimal precision. I looked these macros...

## Standard-compliant way to define a floating-point equivalence relationship

c++,floating-point
I'm aware of the usual issues with floating point arithmetic and precision loss, so this is not the usual question about why 0.1 + 0.2 != 0.3 and the like. Instead, I would actually like to implement a binary predicate in C++ (in a 100% standard compliant way) that actually...

## Confused about assembly FLD instruction m64fp

assembly,floating-point,x86,fpu
I'm so confused. I have some question about the FLD m64fp instruction, but I have no idea where to start. Because this is a homework, I'm not specifically asking for answers, but the method to solve the problem. Any suggestion or idea would be appreciated. Eight consecutive bytes in memory...

## Should exp2 be faster than exp?

math,floating-point
I'm mostly interested in the "exp" and "exp2" functions in C/C++, but this question is probably more related to the IEEE 754 standard than specific language features. In a homework problem I did some 10 years ago, which tries to rank different floating point operations by the cycles needed, the...

## Nasm, not printing the correct value

c,assembly,floating-point,printf,nasm
I've made a nasm procedure that calculates the eucledian distance between two vectors of a certain size. This nasm function is called from a C file which get the result of the function. I've tested, and it works, the value returned is correct, I can print it withoud any problem....

## Converting Float to Int32, with negative numbers fails, C

c,floating-point
I have the following function which is suppose to convert a floating point number to int32. The problem is that for negative numbers it just doesn't work (my if statement isn't executing). I've tried a similar program for a conversion from float to int16 and everything works just fine. Sry...

## fixed point multiplication for normal multiplication

math,floating-point,fixed-point
I need to multiply X with a floating point number in floating point as i don't have floating point operations in my processor. I understand the method but don't know why that method exists? Suppose we want to multiply 2*4.5 in decimal I do the below: 2 * 4.5 (100.1)...

## integer division to float result [duplicate]

java,floating-point,division
This question already has an answer here: Fahrenheit to Celsius conversion 3 answers I have to divide two integers and get a float as result My Code: Float ws; int i = Client.getInstance().getUser().getGespielteSpiele() -Client.getInstance().getUser().getGewonneneSpiele(); int zahl1 = Client.getInstance().getUser().getGewonneneSpiele(); ws = new Float((zahl1 / i)); I check the values with...

## Restrict the input field to enter only numbers or digits - int and float both

javascript,jquery,validation,floating-point,numbers
How to restrict the input field to enter only numbers/digits int and float both. Sometimes we need to allow both integer as well as float value for fields like amount, so in that case the validation is required. There are no of solutions available but they are of large size...

## C99-style hexadecimal floating-point constants in OCaml

floating-point,ocaml,floating-point-conversion
In OCaml, how can I parse C99-style floating-point constants (either as literals or inside strings) in hexadecimal, such as 0x1.b000000000000p4? It seems that they are not valid literals: # let c = 0x1.b000000000000p4;; Characters 12-27: let c = 0x1.b000000000000p4;; ^^^^^^^^^^^^^^^ Error: Unbound record field b000000000000p4 And there seems to be...

## associative-math with GCC

c,gcc,floating-point,floating-point-precision
I have created a double-double data type in C. I tried -Ofast with GCC and discovered that it's dramatically faster (e.g. 1.5 s with -O3 and 0.3s with -Ofast) but the results are bogus. I chased this down to -fassociative-math. I'm surprised this does not work because I explicitly define...

## TCl-Tk How to catch floating point numbers from a file

file,floating-point,tcl,tk
i'm finding some troubles working on a file containing some floating numbers. These are some rows from my file: 174259 1.264944 -.194235 4.1509e-5 174260 1.264287 -.191802 3.9e-2 174261 1.266468 -.190813 3.9899e-2 174262 1.267116 -.193e-3 4.2452e-2 What i'm trying to do is to find the row where is my desire number...

## `std::sin` is wrong in the last bit

c++,matlab,floating-point,ieee-754
I am porting some program from Matlab to C++ for efficiency. It is important for the output of both programs to be exactly the same (**). I am facing different results for this operation: std::sin(0.497418836818383950) = 0.477158760259608410 (C++) sin(0.497418836818383950) = 0.47715876025960846000 (Matlab) N[Sin[0.497418836818383950], 20] = 0.477158760259608433 (Mathematica) So, as far...

## Numpy sum function returns 1.67772e+07

python,arrays,numpy,floating-point,floating-point-precision
I have two big (432*136*136*46) 'numpy.ndarray' H1 and H2 which encompass altitude values corresponding to two simulations. I want to generate an array with 1 when H1 and H2 have the same altitude and 0 when they don't. Then, I want to know how many elements I selected, so I...

## Simple and clean java float to string conversion

java,string,floating-point,formatting,type-conversion
This is a very simple question, but I'm amazed on how difficult it has been to answer. Even the documentation didn't give a clear and straight answer. You see, I'm simply trying to convert a simple float to a string such that the result only has one decimal digit. For...

## Why the division of two integers return 0.00? [duplicate]

c,floating-point,int,printf,format-specifiers
This question already has an answer here: printf(“%f”, aa) when aa is of type int [duplicate] 2 answers Every time I run this program I get different and weird results. Why is that? #include <stdio.h> int main(void) { int a = 5, b = 2; printf("%.2f", a/b); return 0;...

## How to stop JavaScript removing the 0 from a number after the decimal point?

javascript,floating-point,parsefloat
I am using JQuery to do some calculations on some items that a user selects. One of the items which is priced at £13.95, when 2 are selected gives £27.90. However, the result is always displayed as £27.9, the sum removes the last 0. How can I stop the Javascript...

## How to distinguish same int and float elements?

python,list,floating-point,integer
I got following task: create a function that will receive as argument a list of different elements and return a list of original elements (count of each element in list should be not more than 1). Order of elements should remain. Like below: [1, 1.0, '1', -1, 1] # input...

## Is there a standard C way to print floating-point values “perfectly” a la Dragon4?

c,floating-point,floating-point-conversion
Reading Here be dragons: advances in problems you didn’t even know you had I've noticed that they compare the new algorithm with the one used in glibc's printf: Grisu3 is about 5 times faster than the algorithm used by printf in GNU libc But at the same time I've failed...

## Unexpected results for floating point equals

c#,floating-point
Question is not about why 0.1 + 0.9 is not equals 1.0. Its about different behaviour of a equals. Can someone explain why examples below works differently. float q = 0.1f; float w = 0.9f; float summ = q + w; q + w == 1.0f; // False summ ==...

## Is it possible to underflow a floating point addition in C++?

c++,floating-point
I was looking at the Intel Processor manual, volumen 2A, pages 3.266-3.268 and it states that the FADD operation may produce #U (Underflow) exception. The reasoning is that result will be to small to be properly represented in DST. I wondering if addition underflow is possible on C++ using native...

## Convert float to log space in python

python,floating-point,logarithm,natural-logarithm
I am implementing the Viterbi algorithm (a dynamic algorithm) in Python, and I notice that for large input files, the probabilities keep getting multiplied and shrinking beyond the floating point precision. I need to store the numbers in log space. Can anyone give a simple example Python code-snippet of how...

## Is floating point multiplication by zero guaranteed to produce zero?

c++,c,floating-point
I understand floating point has rounding errors, but I'm wondering if there are certain situations where the error does not apply, such as multiplication by zero . Does zero times any number = zero for all floating points ?...

## Is there a way to show correct float value?

python,python-2.7,floating-point,division
Is there a way to show float value in Python 2 like Python 3 does? Code: text = "print('hello, world')" step = 100.0 / len(text) result = 0.0 for _ in text: result += step print result print step print result == 100.0 Python 2.7.9 100.0 4.7619047619 False Python 3.4.3...

## String to Float is not working in java?

java,floating-point
The problem is simple: float f1 = Float.parseFloat("41.975779") //Value for f1 is 41.97578 -> An error of 1ppm And even worse!! float f2 = Float.parseFloat("41.975645") //Value for f2 is 41.975643 -> An error of 2ppm It doesn't matter if I use Float.parseFloat or Float.valueOf, they both give the same result....

## Converting Int to Float loses precision for large numbers in Swift

ios,swift,floating-point,floating-point-precision,floating-point-conversion
XCode 6.3.1 Swift 1.2 let value: Int = 220904525 let intmax = Int.max let float = Float(value) // Here is an error probably let intFromFloat = Int(float) let double = Double(value) println("intmax=\(intmax) value=\(value) float=\(float) intFromFloat=\(intFromFloat) double=\(double)") // intmax=9223372036854775807 value=220904525 float=2.20905e+08 intFromFloat=220904528 double=220904525.0 The initial value is 220904525. But when I...

## how divided integer is converted to floating point number with decimal [duplicate]

java,casting,floating-point
This question already has an answer here: Division of integers in Java 4 answers if 123/33 prints out 3 and 3 is an integer if we cast it to float ( (float)123/33 )how do we get the decimal places from the integer 3. does 3 contains floating points internally...

## Convert list or numpy array of single element to float in python

python,arrays,list,numpy,floating-point
I have a function which can accept either a list or a numpy array. In either case, the list/array has a single element (always). I just need to return a float. So, e.g., I could receive: list_ = [4] or the numpy array: array_ = array([4]) And I should return...

## Assembly - Moving Floating Point Number

assembly,floating-point
I am new in Assembly, I did a lot of searches before asking this but I quite could not understand/find anything I am looking for. fstp dword ptr [eax+00000124] I have this line, so how do I edit it to store any floating number on [eax+00000124], e.g. storing number 6...

## Splitting a floating point number into a sum of two other numbers, without rounding errors

split,floating-point,floating-accuracy
I don't know if it is possible to do this, but i need to split a floating point number in sum of two number... For example assuming x is a floating point number and we want to split this in x = I + f, where I is the signed...

## Numerical issue with np.exp()

python,numpy,floating-point
I have the following code x = -10 for i in range(2,10): print i, " | ",np.exp(-x**i) with the following output: 2 | 3.72007597602e-44 3 | inf 4 | 0.0 5 | inf 6 | 0.0 7 | inf 8 | 0.0 9 | inf Why is the results ~0...

## Check if given number is Even, Odd or Neither in PHP? [duplicate]

php,floating-point,numbers,integer
This question already has an answer here: php test if number is odd or even 11 answers How can I get if a number is even or odd or neither (have decimal, like 1.5) with PHP? I know that there are operators like *, /, but they did not...

## SSE division by integer

assembly,floating-point,x86-64,sse
I am currently working on function, that calculates Taylor approximation of sin(x) function, using C & 64-bit assembly combined (C using asm function). I am moderately new to assembly & low-level programming, and I still don't get few things. Let's call function in C: float taylor(float fi, float n); where...

## Hybrid program (.asm + .cpp): modify small math program's code to include float input

c,assembly,floating-point,hybrid
(Intel x86. TASM and BorlandC compilers, and TLINK used.) In main1.cpp the program takes int input (until you input a number smaller than -999999), puts it into an array x[], puts the number of inputs into array's 0th element, sends array's pointer to f1.asm, adds the numbers, and returns the...

## String to float C#, odd result [duplicate]

c#,string,floating-point
This question already has an answer here: C# String to Float Conversion 5 answers In my code I want to convert a string to a float. But when I conver something like 49.5 to a float, it gives the output 495 instead of 49.5 how can I solve this?...

## Why is this floating point addition result not correct?

I am trying to calculate the time in buffer in microseconds. But I don't understand why the floating-point operation result of my code is not correct. float time, sec; int h, m; sscanf(16:41:48.757996, "%d:%d:%f", &h, &m, &sec); printf("buffer %s\n",buffer); printf("hour %d\n",h); printf("minute %d\n",m); printf("seconde %f\n",sec); time=3600*h+60*m;+sec; printf("%f\n",time); When I execute...

## SIMD minmag and maxmag

assembly,floating-point,x86,sse,avx
I want to implement SIMD minmag and maxmag functions. As far as I understand these functions are minmag(a,b) = |a|<|b| ? a : b maxmag(a,b) = |a|>|b| ? a : b I want these for float and double and my target hardware is Haswell. What I really need is code...

## Why double can store bigger numbers than unsigned long long?

c++,c,floating-point,floating-point-precision,floating-point-conversion
The question is, I don't quite get why double can store bigger numbers than unsigned long long. Since both of them are 8 bytes long, so 64 bits. Where in unsigned long long, all 64 bits are used in order to store a value, on the other hand double has...

## unsupported operand types for * : 'float' and 'decimal'

python,floating-point,decimal
for a in range(0,size): et = 0.0023*ralist[rows[a][2]] * ( 0.5*(rows[a][3] + rows[a][4]) + 17.8 ) * ( rows[a][3] - rows[a][4])**(0.5) eto_values.insert(a,et) When I try to run the code, I get the following error: unsupported operand types for * : 'float' and 'decimal' I have tried using decimal.Decimal() function also. Can...

## Float comparision in C [duplicate]

c,if-statement,compiler-errors,floating-point,floating-point-precision
This question already has an answer here: How dangerous is it to compare floating point values? 8 answers #include<stdio.h> int main() { float x = 0.6; if (x == 0.6) printf("IF"); else if (x == 0.6f) printf("ELSE IF"); else printf("ELSE"); } This code gives output ELSE IF #include<stdio.h> int...

## xmm, cmp two 32-bit float

assembly,floating-point,sse
I'm trying to understand how to compare two floating point numbers (32-bit) using the xmm registers. To test I've written this code in C (which calls the code in assembly): #include "stdio.h" extern int compare(); int main() { printf("Result: %d\n", compare()); return 0; } Here is the assembly, I want...

## Floating point math in python / numpy not reproducible across machines

python,numpy,floating-point,blas
Comparing the results of a floating point computation across a couple of different machines, they are consistently producing different results. Here is a stripped down example that reproduces the behavior: import numpy as np from numpy.random import randn as rand M = 1024 N = 2048 np.random.seed(0) a = rand(M,N).astype(dtype=np.float32)...

## Why are the programming languages limits precision of 22/7 equally?

I tried Erlang \$ erl 1> Pi = 22/7. 3.142857142857143 Haskell \$ ghci Prelude> 22/7 3.142857142857143 Python \$ python >>> 22/7.0 3.142857142857143 Ruby \$ irb 2.1.6 :001 > 22 / 7.0 => 3.142857142857143 The result is the same. Why?...

## handling float arguments python

python,error-handling,floating-point
I have a function that takes an optional argument like so: myProgram -n 8 I want to add in error handling that will exit the program and print an error message if the argument that the user enters is a float. How would I test for this if the argument...

## How do you convert float to a decimal integer times a power of 2?

floating-point
For example taking 3.5625 and representing it as such -> 57 * 2^-4. If anyone can point me in the right direction, it would be very helpful. Thanks in advanced!

## More cttypes c_float() precision

python,floating-point,ctypes,floating-accuracy
I'm using Python's ctypes library to call my C code. My problem is that when I try to create a c_float, I seem to obtain a slightly different value to what I set. For example print(value) print(c_float(value)) 0.2 c_float(0.20000...298...) How can I avoid this?...

## Convert “float;#22.0000000000000” into just “22.0” in C#

c#,sharepoint,floating-point
I'm doing a ToString() on an object and the output I am getting is float;#22.0000000000000. I just need 22.0. How do I achieve this in C#? Additional info: The object is the value from a Number column in a SharePoint list. The value is being retried in my code with...

## Is floating point precision mutable or invariant?

binary,floating-point,decimal,floating-point-precision,significant-digits
I keep getting mixed answers of whether floating point numbers (a.k.a. float, double or long double) have one and only one value of precision, or have a precision value which can vary. One topic called float vs. double precision seems to imply that floating point precision is an absolute. However,...

## why (int) ( 307.03 * 100) = 30702 [duplicate]

php,types,floating-point,int
This question already has an answer here: Is floating point math broken? 18 answers Today my coworker stumbled on this: \$s = floatval("307.03"); \$s = \$s * 100; echo intval(\$s); //30702 float value or round(\$s) return 30703 as expected. I guess it's a problem connected with float to int...

## Using FMA (fused multiply) instructions for complex multiplication

c++,floating-point,fma
I'd like to leverage available fused multiply add/subtract CPU instructions to assist in complex multiplication over a decently sized array. Essentially, the basic math looks like so: void ComplexMultiplyAddToArray(float* pDstR, float* pDstI, const float* pSrc1R, const float* pSrc1I, const float* pSrc2R, const float* pSrc2I, int len) { for (int i...

## Converting a string holding binary to float

c++,string,floating-point
I have a string holding a value in binary that I want to convert to float. I can't find a way to do this. for example, I have a string temp = "00000000000000000000000101111100"; to represent 0.25 in binary. Using stof on temp with string::size_type yields 1.0111110e+008 stored in the float...

## Dynamic Memory Allocation to Reverse Print Array of Floating Point Numbers

c,arrays,struct,floating-point,malloc
So I'm brand new to C and playing around with memory allocation for arrays. I'm trying to create a program that will dynamically allocate space using malloc to reverse an array of floating point numbers. #include <stdio.h> #include <stdlib.h> struct Rec { float * x; int size; }; int main(){...

## Algorithm for Floating point conversion format IEEE

floating-point,floating-point-conversion
Assuming i have a floating point (fp) of a given format FA (i.e. with his exponent size, mantissa size), and more specifically something like FA fa; and suppose i wanted this to a format FB with an operation FA2FB, which gives a floating point number fb, i.e. something like FB...

## Percentage of various Floating Point Operations in SPEC CPU Benchmarks

c++,math,floating-point,fortran,benchmarking
I am looking at the SPEC CPU2006 benchmark website for floating-point: SPEC 2006 Floating Point I noticed that all of the benchmarks are listed but I couldn't find any information in regards to the percentage of basic floating point operations such as add/sub, mult, sqrt, div, etc. How would I...

## f:convertNumber unexpectedly increments/decrements float value

jsf,jsf-2,floating-point,converter
I have a float property bound to <h:inputText>. For long values, it was automatically converting the values to exponential notation. I tried to use <f:convertNumber> to avoid the exponential value presentation. The value can be of two given below. Format 1: <18 digits> Format 2: <14 optional digits>.<3 option decimals>...

## Why are some float < integer comparisons four times slower than others?

python,performance,floating-point,cpython,python-internals
When comparing floats to integers, some pairs of values take much longer to be evaluated than other values of a similar magnitude. For example: >>> import timeit >>> timeit.timeit("562949953420000.7 < 562949953421000") # run 1 million times 0.5387085462592742 But if the float or integer is made smaller or larger by a...

## How can I print decimal places, up-to the point there are not two repeating zeros?

java,floating-point,formatting
I would like to change the below output 5250.000000000000 5512.500000000000 5788.125000000001 6077.531250000001 6381.407812500002 6700.478203125002 7035.502113281253 7387.277218945315 7756.641079892581 8144.47313388721 To the below output. 5250.0 5512.5 5788.125 6077.53125 6381.4078125 6700.478203125 7035.50211328125 7387.277218945313 7756.641079892578 8144.473133887207 So it appears the logic I need, is to only print decimal places if there is less than...

## Exact decimal arithmetic in Julia

floating-point,julia-lang,multiplication
Due to the nature of floating-point math, .4 * .4 = 0.16000000000000003 in Julia. I want to get the mathematically correct answer of 0.16, in a CPU-efficient way. I know round() works, but that requires prior knowledge of the number of decimal places the answer occupies, so it isn't a...