I have a large number of inserts and updates coded in my program that involves money, and I cam across a serious rounding bug. The majority of the time, I want to truncate the values, instead of the SQL Server default of rounding. Rather than changing all my existing code,...

Can someone help me explain this code that is converting decimal fractions into a binary? Convert the decimal fractions into a binary form: x = float(raw_input('Enter a decimal number between 0 and 1: ')) p = 0 while ((2**p)*x)%1 != 0: print('Remainder = ' + str((2**p)*x - int((2**p)*x))) p +=...

I'm building a financial website with the Flask framework and I'm currently writing unit tests for it. I'm using the Babel package to format money amounts and I hit upon rather strange rounding behaviour. I would expect rounding to be up in case of a 5, or to at least...

I'm trying to find the best column type to use for numeric values of varying length both before and after the decimal place. The DECIMAL type seems to only allow a fixed-length number with a fixed-length also after the decimal. I need something that preserves to exact precision numbers like:...

My calculations seem to be showing as incorrect now despite the decimal format being correct. import time # It was reccommend in our text to always have your import data located outside of any loops, thus I put mine here. while True: # This is the main, and only; while...

How can a value such as "1.5" be passed in a a URL POST request using swift? For example: let number = "1.5" let numberValue = number.stringByAddingPercentEscapesUsingEncoding(NSUTF8StringEncoding) let server:String = "www.someserver.com" let phpFile:String = "/php/SomePHPScript.php" let baseURL = NSURL(string: "http://\(server)\(phpFile)") let url = NSURL(string: "?value=\(numberValue)", relativeToURL: baseURL) let cachePolicy =...

NB: To the best of my knowledge this question is not a duplicate! All the questios/answers I found are either how to eliminate points from data that are already in R or how to change the decimal point to a comma when loading it. I have a csv with numbers...

I have three numbers in decimal form, they make itself one more decimal number. <120, 111, 200> - (120 * 256 + 111) * 256 + 200 = 7892936 - Decimal. I keep the number is because I have a variable number of bytes to write the number. Q: How...

I am trying to export a CSV file with a wrong content type so it gets opened in Excel. When the value is 2.0 from object, in Excel sheet it displays only 2. It just omits the decimal value. It works fine when the value is 2.1, 2.2, etc. The...

So , i have a problem . i am calculating the discount and it may be in decimal points . the method i am using is not working for the value 0.0038575 and the method is value=0.0038575 Math.round(value * 100) / 100, it is returning the same value . this...

How can I copy a DataFrame to_cliboard and paste it in excel with commas as decimal? In R this is simple. write.table(obj, 'clipboard', dec = ',') But I cannot figure out in pandas to_clipboard. I unsuccessfully tried changing: import locale locale.setlocale(locale.LC_ALL, '') Spanish_Argentina.1252 or df.to_clipboard(float_format = '%,%') Thanks in advance...

I have the following public decimal? Price {get;set;} When I enter 3000.00 in to the textbox on the view (textbox below) <div class="form-group"> <label class="col-lg-3 control-label no-padding-right">Price</label> <div class="col-lg-5"> @Html.ValidationMessageFor(m => m.Price) <div class="input-group"> <span class="input-group-addon">£</span> @Html.TextBoxFor(m => m.Price, new { @class = "form-control", type = "text", id = "txtPrice",...

I've issues rounding numbers while calculating tax. for example for amount 2.12 and a tax of 20%, I calculate it like follows: # this is on Numeric class def ceil_money (self * 100).ceil / 100.0 end ... amount = "2.12" ((Float(amount.to_f) * Float("20.0"))/100).ceil_money result is 0.43. To paypal I have...

When programming in C, say if I have integer h as a hexadecimal value and integer d as a decimal number. Can I do addition or subtraction between h and d? Or do they have have to be in the same number system?

I am trying to write a calculator in c++ that does the basic functions of /, *, -, or + and show the answer to the hundredth decimal place (0.01). For example 100.1 * 100.1 should print the result 10020.01 but instead i get -4e-171. From my understanding this is...

Is there anyway to have perl run through a file of numbers and print out only the numbers that don't have a decimal point in them? So lets say I have 60, 78.9, 109 and I want it to print out 60, 109. Is this possible in Perl?...

I'm using the division obtained from using from __future__ import division in Python 2.7, and I'm using the complex numbers in Python. I want to get more decimals from the division of complex numbers. Does exists a way to do this? I know the decimal.Decimal command, but it needs to...

I have a Magento webshop, available in Dutch (NL) and English. There are different products, but all cost 1.00 euro. I managed to get the price to 1.00 in English and 1,00 in Dutch. Now, what I want is to edit the decimals in Dutch. I want to display 1,00...

im not sure if m using the wrong data types or the wrong instruction or what but div,idiv,fdiv all seem to give me a 0 for the quotient. And i want to get some decimal number rounded to a couple decimal places. i think im using the correct registers for...

This question already has an answer here: What's the best way to get the fractional part of a float in PHP? 6 answers How to get whole and decimal part of a number? 6 answers I have variable which is a float number $num = 10.5; and I want...

In my C# application, I am trying to save a decimal price into a SQL Server table. Columns type is decimal with no total digits defined. Without discount calculations, everything works fine. But when I run some calculations, I get a final value of 21800, and I get the following...

This is a question about precision, rounding, data types, and using the xs:integer() function in XSLT 2.0. We have the following line of code: <xsl:value-of select="xs:integer(round(InvoiceAmount * 100))"/> When the value of InvoiceAmount in the source XML is -62.84, the result of the value-of is -6283 which is incorrect. We...

I have Sharepoint list values that are Currency, such as: list.Fields.Add("Airfare", SPFieldType.Currency, false); But how do I save these values into the list as the Currency type? I tried this: spli["Airfare"] = (SPFieldType.Currency) boxAirfare.Text; ..and this: spli["Airfare"] = boxAirfare.Text as SPFieldType.Currency; ...but they won't compile, telling me, "'Microsoft.SharePoint.SPFieldType.Currency' is a...

I am trying to create a mapping using Fluent NHibernate with Oracle 11g. This is the mapping I am creating:- public class Abc : ClassMap<Abc> { public Abc() { Table("Abc"); DynamicUpdate(); Id(x => x.Id, "IdColumn").GeneratedBy.GuidComb(); Map(x => x.DecimalColumn, "DecimalColumn").Formula("TRUNC(DecimalColumn, 28)"); } } Now when I get the data using criteria...

I have an integer value for example 1171906 and i want it to be display like this 1,171,906 Please give me some easy way in jsp code. Thanks in advance....

I am currently working in Microsoft Visual Studio Express 2013 with a SQL Server back end. I am using a SQL query to pull a datatable and then selecting the first cell to pull a number from. I need to take this number from the datatable and then change a...

I'm trying to write some quick code to calculate different orders of magnitude of the number 10. Should be pretty simple, but the results are a little bit odd. When calculating 10^-6 things a lot of extra decimals get added. The results look a bit like this: 0.1 0.01 0.001...

If I have a decimal number representing an angle and I add it to another angle, how can I ensure that it remains between 0° and 359°? So if it's 120° + 270°, it will be 30° instead of 390°? I could check if they are above or below the...

Computers in my country do not accept decimals with a '.' they only accept ',' How do I write a program which would accept 12.5 from the console as a decimal number?...

I have a SQL Server stored proc which I'm calling from C#. Some of the parameters are Decimal, scale and precision vary but most often these are scale 8 and precision 2. So I need to use the Decimal dbtype for these. ALTER PROCEDURE [dbo].[SaveAssortmentPlanItem] @hierarchy nvarchar(13), @closing_store_stock nvarchar(10), @closing_westway_stock...

I am new to WPF. I have a requirement that I need to develop a custom textbox control which should support the functionality like: Should accept only decimal values. Should round off to 3 decimal places when assigned a value through code or by the user. Should show the full...

I am new to Perl. I have two different sets of arrays. Those two arrays having decimal values, both does not have an exact matches, it might be differ only either +0.5 or -0.5. I tried this program, it's not working. @inpt = qw(1003.3965 1036.4392 1037.3843 1045.4459 1101.4259 1107.4253 1118.3928...

How do you fix the following problem converting from decimal to binary? void tobinary(int bin) { string binary = Convert.ToInt32(bin, 2); } These are the errors: Error 2: Argument 2: cannot convert from 'int' to 'System.IFormatProvider' 42 Error 1: The best overloaded method match for 'System.Convert.ToInt32(object, System.IFormatProvider)' has some invalid...

Im trying to make my c program run any value that you put in will come out as words for example if I wanted to put in the value 1234.56 it should come out as "One Thousand Two Hundred Thirty Four Dollars ... and 56 Cents" Notice how the cents...

I want to create a function, which has parameters with value, decimal mark and number of digits after decimal mark. This function will convert double value to string with given parameters. Here is the function I created. public static string ConvertDoubleToString(double value, string decimalMark = ".", int decimalDigits = 2)...

I have Hex string like 1480D604. Which I need to change order from Low to High 0x04D68014 and then I need to cast it into decimal value. One approach that I can think is first change their order first like. Step 1 : 14-80-D6-04 --> 04-D6-80-14 How can I change...

The below code is used to count number of digit in a given decimal. The problem is that it doesn't count digits more than 10. int NumDigits(int n) { int digits = 0; if (n <= 0) { n = -n; ++digits; } while (n) { n /= 10; ++digits;...

So I need a Regex to match the ASCII character STX(Start Of Text) which is number 2 on ASCII Table. I tried ^ as Regex101.com suggest for start of text but it was unsuccessful. Then I tried [[:ascii:]] which was matching for all ASCII characters. Is there any way to...

user enters a number in either hex or binary (must be prefixed with 0x or 0b, if not gives the error message). read user input as a string i can't use radix, i'm just using integer.parseInt(s) i am not sure how to make this program work.. please help what am...

I am not sure how to approach it but could someone help me convert the following numbers to their decimal representation: and ...

I'm having problems with error: input string was not in a correct format. I'm trying to convert curency in datagrid. At point where I get error (this is where I set value to value variable) text variable haves value 22.22 so I don't know what is wrong with format. public...

char str[20]; printf("Enter anything\n"); scanf("%s",str); for(int i = 0 ; i<strlen(str) ; i++) { if( isdigit(str[i])) printf("%d",(int)str[i]); } I tried with the above code but the output is in ASCII values of the numbers rather then actual numbers. Like , input: "0" output: "48". When I try (int)(str[i]-'0'); instead of...

I'm working on a AutoIt script and I have to convert some weird hexadecimal number to normal integer. But I don't know which procedure should I follow to convert my hex number. For example, in the AutoIt documentation below, some definitions and their hex values are given. But they didn't...

I have the following working code; it accepts a string input as the function parameter and spits out the same string converted to a decimal. I'm not going to bother accounting for negative inputs, although I understand that I can set a boolean flag to true when the first indexed...

I want to compare two decimals in c# with some margin of error. Can anyone point out problem with the following code. Note that I am interested in 6 decimal places after that I could ignore the values. var valOne = decimal.Round(valueOne, 6); var valTwo = decimal.Round(valueTwo, 6); var difference...

I'm trying to get my output to print to only 1 decimal place.I've tried using the DecimalFormat but it is messing up the alignment of my output. Here is what my output was before I used the DecimalFormat: ID Programs Midterm Final Weighted Average -- -------- ------- ----- -------- -------...

I'm probably missing something that is really, really simple, but I can't for the life of me figure out what it is that I'm not doing correctly... I have this query which is used to pull out hours people have completed in volunteering and then assign them an award based...

As I am dealing with very small numbers, i changed the operation (1) to (2). After this transformation, the time-cost is the same but when I try to improve precision using decimal.Decimal and transforming (2) to (3) the time-cost explodes (~ 10 times ). Is there a solution to solution...

I am making a table that is pulling data dynamically from a JSON file and I need to be able to change the value to be at two decimal places. Is this even possible based on the code I already have? Examples of what I want to do: --7 changes...

I'm working on a project that uses decimals in a textbox. I'm currently developing in a machine that has decimal separators set to "," instead of "." so I had to use this sentence when parsing text string into decimal: decimal number = Decimal.Parse(number.Text, CultureInfo.InvariantCulture); Now... Is CultureInfo.InvariantCulture the right...

Why output of following code is in scientific notation? BigDecimal val = new BigDecimal("0000.000000111"); System.out.println(val); output: 1.11e-8 but output of this code is proper in decimal format. BigDecimal val = new BigDecimal("0000.000011111"); System.out.println(val); output: 0.000011111 Is there a way to get proper decimal value for first string(like second output)?...

I have a list list1 = ["1.10","1.1","1.23","2.20","2.2"] and I used list1.sort(key=float) which gave output list1 = ["1.10","1.1","1.23","2.20","2.2"] However I am looking for this: list1 = ["1.1","1.10","1.23","2.2","2.20"] Any ideas how I can go about doing this?...

I'm working on a number of different ways to use variables in functions, but there is one way I just can't figure out after about an hour of searching the web. I just want the user to be able to enter a decimal number like "23.45" and the program print...

Maya commands library seems to dislike working with the built in decimal module in python. I am trying to scale an object by an arbitrarily large float for precision. As far as understanding that you can't scale by a decimal 'object' is clear. ive been trouble shooting the math and...

I want only 2 decimals in my code. How can i do that? Here is my coding: double conta1 = ((num1*35)/100)*1.05; bomba1.setText(Double.toString(conta1)); Num1=55 Well i only want to result to be 20.21 instead of 20.212500000... How can I do that?...

I want to calculate with numbers having 3 places before and 2 places after decimal point (of course, 2 and 3 are configurable). I think it will be easiest to explain by examples: 0.01 and 999.99 are the lower and upper limit for positive numbers. Of course, there is also...

i'm using the latest Breezejs (1.5.3) for data operations in sql server with Entity Framework. The save operation is done successfully. the savemap has the decimal value with full scale but after update the field in DB has wrong scale (number of decimal points). e.g the savemap has the value...

Suppose that we have this code: a = pi; b = pi+2; c = pi*2.2 out = [a b c] returns: out = 3.1416 5.1416 6.9115 I want this output (without rounding values to two decimal): out = 3.14 5.14 6.91 I don't want print these values. I want see...

Hello I have a textfile with (ASCII or other format i don't know maybe UTF or UNICODE) text. In this file are special letters. I want to convert all letters, the complete file into DECIMAL. I just don't know how. private void simpleButton1_Click(object sender, EventArgs e) { int size =...

I have in database product prices with 4 digits in decimal part, i.e; 4.5000 0.0050 5.0000 I want to show on the website these prices with minimum 2 decimal digits (without rounding), i.e; 4.50 0.005 5.00 I tried number_format, but it still leaves 4 digits in decimal part. And also...

am new in c# so how to replace the string for example:- label1.Content = "Bal.Rs." + 100; How to get the 100 only while we save from label1.Text???...

I'm trying to find solutions to a simple MDAS (i.e. Multiplication, Division...) problem using C#, and while it mostly gets correct solutions, I have problems when the variables add up to x.99999... so I get wrong answers because it doesn't compute to 1. For example, if I have: decimal a...

I have this number 1.2345679E6 How can I format it to 1.23E6 using DecimalFormat? DecimalFormat df = new DecimalFormat("#.##"); ...

I'm new to C# (Come from Java/C++ at uni so it's not really that new I guess) but for a project I'm needing to compare decimals. e.g. a = 1234.123 b = 1234.142 Decimal.Compare() will of course say they're not the same as a is smaller than b. What I...

for a in range(0,size): et = 0.0023*ralist[rows[a][2]] * ( 0.5*(rows[a][3] + rows[a][4]) + 17.8 ) * ( rows[a][3] - rows[a][4])**(0.5) eto_values.insert(a,et) When I try to run the code, I get the following error: unsupported operand types for * : 'float' and 'decimal' I have tried using decimal.Decimal() function also. Can...

I have a method that is converting a decimal (double value) into a fraction and putting the numerator and denominator values into an int[] of size 2. Testing it works out fine for most values except when I hit 0.0001. Then the return value is 1.0/1.0. The method: private static...

The default decimal points are 4 digits "1.0000" is there a way i can adjust it on front end that would only show 2 decimal points? like: "1.00" Thanks!...

I am making a converter from GPS Degrees, Decimal Minutes to Degrees, Minutes, Seconds since I have no found one on the internet. Can you please tell me how to get rid of the numbers after a decimal in the calculated total field? <HEAD> <TITLE>Javascript Calculator</TITLE> <SCRIPT language="javascript" type="text/javascript"> function...

I am dealing with a bug where applying the str function to an instance of decimal.Decimal gives me the string '0E-1000'. I expected that str applied to a Decimal would always give me a string of the form -?[0-9]+(\.[0-9]+), which is what I want. Firstly, if I understand correctly, the...

I have to convert big numbers in perl from decimal to binary and the other way around. An example number of that length: Dec: 76982379919017706648824420266 Bin: 111110001011111001010101000010011001000010101111001110000000000000000000000000000000000000000000 In the forums i found two functions: sub dec2bin { my $str = unpack("B32", pack("N", shift)); $str =~ s/^0+(?=\d)//; # otherwise you'll get...

How to set space as thousand separator for DecimalFormat in float? I want to show 13,52 as 13,5, but 13,00 as 13 and I did this with new DecimalFormat("#.#").format(fIncome) but I want to 1400,5 be 1 400,5 and 1400 to be 1 400. For double I use this code (I...

I've come across two different precision formulas for floating-point numbers. floor( (N-1) Log10(2) ) = 6 decimal digits (Single-precision) and Log10(2N) ≈ 7.225 decimal digits (Single-precision) Where N = (24) Significant bits (Single-precision) The first formula is found at the top of page 4 of "IEEE Standard 754 for Binary...

I have made a datatable "Event_Table" with 46 rows and 6 columns. At some point I export this to text file and would like the output of some fields to be truncated to exactly 2 decimal places. Event_Table[1:34,3:6]=round(Event_Table[1:34,3:6]) Event_Table[36:39,3:6]=format(round(Event_Table[36:39,3:6],2), nsmall=2) Event_Table[41:46,3:6]=format(round(Event_Table[41:46,3:6],2), nsmall=2) Line 1 and 2 produce the desired result,...

I keep getting mixed answers of whether floating point numbers (a.k.a. float, double or long double) have one and only one value of precision, or have a precision value which can vary. One topic called float vs. double precision seems to imply that floating point precision is an absolute. However,...

I'm sure I'm missing something here. Take a look: On my MySQL database, the 'value' field is a decimal, like this: My database enconding is: utf8_general_ci. My PHP code, to show the values: $sale_limit_gen = Coo::getSaleLimitById(1); $sale_limit_ale = Coo::getSaleLimitById(2); echo $sale_limit_gen['value']; echo '<br>'; echo $sale_limit_ale['value']; echo '<br>'; echo $sale_limit_ale['value'] *...

I want to convert a decimal a with scale > 0 to its equivalent decimal b with scale 0 (suppose that there is an equivalent decimal without losing precision). Success is defined by having b.ToString() return a string without any trailing zeroes or by extracting the scale via GetBits and...

I'm trying to code a program that returns the golden ratio up to a limit of decimal places. What I have so far is this: def gr(n): a, b = 0, 1 for x in range(0, n): a, b = b, a+b else: return(a) decimals = int(input("What amount of decimals...

So I am trying to make a script where the user inputs the number of miles they need to travel and the miles per hour they are traveling and the script outputs the hours and minutes leftover that they must travel. I've been trying to use % to find the...

This question already has an answer here: What does it mean when a numeric constant in C/C++ is prefixed with a 0? 7 answers printf with “%d” of numbers starting with 0 (ex “0102”) giving unexpected answer (ex '“66”) 3 answers With this code snippet: int a = 011;...

can anyone please tell me why this : declare @decimal decimal(4,3) = 10 gaves me an error in SQL SERVER : msg 8115 level 16 state 8 line 1 arithmetic overflow error converting int to data type numeric even this declare @decimal decimal(4,3) = 10.0 gave this error : msg...

I text box i have a value: 1212,12 Additionally my text box have a mask: MaskType=Numeric EditMask='n2' i try to parse it: var culture = CultureInfo.CreateSpecificCulture("en-US"); var value = decimal.Parse(myTextBox.Text, culture); but get value=121212 when expected value=1212.12 What can be wrong?...

I'm developing a change history table where I'll basically record the old and new value for changes in fields of two types: decimal and datetime. To make it simple, I was thinking about create a string field and convert the values to string before store in the table. My problem...

I'm looking for a solution to auto increment decimal value in a column until maximum value has reached for each column. I have managed to make an formula for column A { =SUM(B3+0,1) } By dragging down cell 1 to a maximum value that i require, however this is not...

This question already has an answer here: arr.sort() does not sort integers correctly 8 answers I can't seem to figure this one out. var arr = [2.62, 111.05, 1.05] arr.sort(); This returns [1.05, 111.05, 2.62], but I'm expecting [1.05, 2.62, 111.05]. How can this be achieved? I've read a...

I want seven digit after dividing a integer,i have ddhh.mmmmm lat long I have to convert it dd.(hhmmmm/60) with 7 digit after point. My code: public static String getCoordinates(String data) { System.out.println(data); String mm = data.substring(2).replace(".", ""); int int_mm = Integer.parseInt(mm); double d_mm = int_mm / 60.00000000; String s_mm =...

When i run this code it gives NumberFormatException? how do i convert A49D43C5 this numbers to decimal in java? logger.info("Decimal value: " + Integer.parseInt("A49D43C5", 16)); ...

class simple { public static void main() { double n, m; Scanner ip = new Scanner(System.in); n = ip.nextDouble(); m = ip.nextDouble(); m = n * m; System.out.println("Value in " + m); } } In the above code.. My output should only use decimal poinf if my input is using...

I am working on android application in which i am using seek bar. In which my range is from 0 - 20 and starting value is 5. For numeric values it is working fine, but now i want to make it for decimal values but the range should be from...

I am using Python 2.7 and MySQLdb. I am trying to update and set as the number the decimal I have set as data but what I get is an the closest integer. This is the code: Value = 5 data = 5 data = data + 0.5 print(data) x.execute("""...

I will begin by saying I know basically nothing about VBA and am just trying to create a spreadsheet for work. I'm trying to make a VBA module to sum cells excluding cells with a strikethrough. The code I'm using below works great, but will not calculate decimals. For instance,...

Let's say I have a string (like a fraction); var num = "1/2"; Why does this work: var y = num.split(' '); if (y.length > 1) { var z = y[1].split('/'); var a = (+y[0] + (z[0] / z[1])); num = a; } else { z = y[0].split('/'); if (z.length...

I am having a problem with encoding to JSON in Python, specifically with decimal.Decimal values. I am using this to output JSON for a Google App Engine application. To circumvent the exception from the default json module in Python telling me that it can't handle decimal.Decimal objects, I am using...

I would like to plot some data as a function of time. I have used matplotlib and datetime, and it works well except for the x axis ticks. The time interval is about an hour, and there are ticks every 10 minutes. However, the ticks do show, for example, 17:50,...

Morning, I currently have a SELECT statement calculating the PCT for a sports table: SELECT team , COUNT(*) as played , SUM(win) as wins , SUM(loss) as lost , SUM(draw) as draws , SUM(SelfScore) as ptsfor , SUM(OpponentScore) as ptsagainst , SUM((win*2 + draw)- loss) as score , CAST(SUM(win +...

I need a function to check if number have 2 decimals or not. For example: $number = '1.00'; // Valid $number2 = '1'; // Not valid $number3 = '1.000' //Not valid ...

I have been playing with DateTime in C#, and have come up with some useful code for calculating days between dates. I use the following code to work out how many days there are between the dates, then I divide the number of days by 7 to work out how...

How can I calculate the position of the first significant digit in a fraction? Here are a few example inputs with desired output 0.123456 // 1st sign. number position = 1 0.0012345 // ... = 3 0.000012345 // ... = 5 ...

ok I'm very new to R and this is a general question. I have various numbers and I want to replace the whole part and keep the decimal part unchanged. For example I have values from 25.01 to 25.99, I want to replace 25 with 10 but keep the decimal...

I am currently trying to figure out how to make the correct conversion from two's complement hexadecimal to its base 10 equivalent. We were given an assembly language program that adds and subtracts 32-bit integers and need to take the values of the registers and convert them. Here is what...

I made a KeyPress Event and want to allow only Double values (or just digits and comma) so I tried this: e.Handled = !(char.IsNumber(e.KeyChar) || e.KeyChar == (char)Keys.Back || e.KeyChar == (char)Keys.Decimal); But somehow he's got problems with the "Decimal". I'm using a german keyboard and when I try to...