Precision error?

21 Dec 2005 6:38 AM

tym

Hi,

I have written a simple code as follorws:

//************* BEGIN CODE ******************
double a;
a= 106.3 - 137.3
//************* END CODE ******************

I am expecting a value of -31.

But, I get a value of -31.000000000000014

Did I miss out something here? Due to system design, we do not wish to use
the 'round' command.

Can anyone enlighten me whether this is a bug?

Thanks
rgds,

21 Dec 2005 6:45 AM

Michael Nemtsev

Hello tym,

Why not to use this?

decimal a;
a = 106.3m - 137.3m;

t> I have written a simple code as follorws:
t> //************* BEGIN CODE ******************
t> double a;
t> a= 106.3 - 137.3
t> //************* END CODE ******************
t> I am expecting a value of -31.
t> But, I get a value of -31.000000000000014

---
WBR,
Michael Nemtsev :: blog: http://spaces.msn.com/members/laflour

"At times one remains faithful to a cause only because its opponents do not
cease to be insipid." (c) Friedrich Nietzsche

21 Dec 2005 6:54 AM

tym

Hi Michael,

The example is a simplified scenario. In my programs, the value could be
obtained from databases or other sources. So, it could be something like:

double a;
a = var1 + var2

where var1 and var2 could be passed in from some external source.

Thanks
rgds,

Show quote

"Michael Nemtsev" wrote:

> Hello tym,
>
> Why not to use this?
>
> decimal a;
> a = 106.3m - 137.3m;
>
> t> I have written a simple code as follorws:
> t> //************* BEGIN CODE ******************
> t> double a;
> t> a= 106.3 - 137.3
> t> //************* END CODE ******************
> t> I am expecting a value of -31.
> t> But, I get a value of -31.000000000000014
>
> ---
> WBR,
> Michael Nemtsev :: blog: http://spaces.msn.com/members/laflour
>
> "At times one remains faithful to a cause only because its opponents do not
> cease to be insipid." (c) Friedrich Nietzsche
>
>
>

21 Dec 2005 7:07 AM

Jon Skeet [C# MVP]

tym <t**@discussions.microsoft.com> wrote:
Show quote

> I have written a simple code as follorws:
>
> //************* BEGIN CODE ******************
> double a;
> a= 106.3 - 137.3
> //************* END CODE ******************
>
> I am expecting a value of -31.
>
> But, I get a value of -31.000000000000014
>
> Did I miss out something here? Due to system design, we do not wish to use
> the 'round' command.
>
> Can anyone enlighten me whether this is a bug?

No, it's not a bug. See
http://www.pobox.com/~skeet/csharp/floatingpoint.html

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

21 Dec 2005 9:03 AM

tym

Ok, thanks Jon

rgds,

Show quote

"Jon Skeet [C# MVP]" wrote:

> tym <t**@discussions.microsoft.com> wrote:
> > I have written a simple code as follorws:
> >
> > //************* BEGIN CODE ******************
> > double a;
> > a= 106.3 - 137.3
> > //************* END CODE ******************
> >
> > I am expecting a value of -31.
> >
> > But, I get a value of -31.000000000000014
> >
> > Did I miss out something here? Due to system design, we do not wish to use
> > the 'round' command.
> >
> > Can anyone enlighten me whether this is a bug?
>
> No, it's not a bug. See
> http://www.pobox.com/~skeet/csharp/floatingpoint.html
>
> --
> Jon Skeet - <sk***@pobox.com>
> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
> If replying to the group, please do not mail me too
>