Home All Groups Group Topic Archive Search About

Query Analyzer and EM display different results

Author
18 Jan 2005 11:51 AM
Phil Cook
When I retrieve a 'float' type value in QA it sometimes displays as
2768.8400000000001 but when viewing the table in EM it shows as 2768.84,
which I believe to be the correct value.

What causes this?


MSSQL 2000
QA v 8.00.194

Author
18 Jan 2005 1:26 PM
Andrew J. Kelly
Float is an approximate value to begin with.  The gui that is displaying the
value may round it off to suite it's needs and I believe that is what you
are seeing.  If you want accuracy to that level use decimal and not float.

--
Andrew J. Kelly  SQL MVP


Show quote
"Phil Cook" <Phil C***@discussions.microsoft.com> wrote in message
news:1D95D27B-AD34-4666-9012-704E229BA585@microsoft.com...
> When I retrieve a 'float' type value in QA it sometimes displays as
> 2768.8400000000001 but when viewing the table in EM it shows as 2768.84,
> which I believe to be the correct value.
>
> What causes this?
>
>
> MSSQL 2000
> QA v 8.00.194
Author
18 Jan 2005 1:41 PM
Phil Cook
Thanks Andrew. I could understand if it was a rounding issue, but I'm pretty
sure the value was only input with 2 dec places. How can the value have been
changed?



Show quote
"Andrew J. Kelly" wrote:

> Float is an approximate value to begin with.  The gui that is displaying the
> value may round it off to suite it's needs and I believe that is what you
> are seeing.  If you want accuracy to that level use decimal and not float.
>
> --
> Andrew J. Kelly  SQL MVP
>
>
> "Phil Cook" <Phil C***@discussions.microsoft.com> wrote in message
> news:1D95D27B-AD34-4666-9012-704E229BA585@microsoft.com...
> > When I retrieve a 'float' type value in QA it sometimes displays as
> > 2768.8400000000001 but when viewing the table in EM it shows as 2768.84,
> > which I believe to be the correct value.
> >
> > What causes this?
> >
> >
> > MSSQL 2000
> > QA v 8.00.194
>
>
>
Author
18 Jan 2005 3:05 PM
Mike Epprecht (SQL MVP)
Hi

float is an approximate data type. Not all values can be held correctly.

In your case, 2768.84 gets stored as 2768.8400000000001 as it is not
possible to represent 2768.8400000000000 in the float data type.

As Andrew said, use Decimal data type if you don't want SQL Server to store
a 'close enough' value.

Regards
Mike

Show quote
"Phil Cook" wrote:

> Thanks Andrew. I could understand if it was a rounding issue, but I'm pretty
> sure the value was only input with 2 dec places. How can the value have been
> changed?
>
>
>
> "Andrew J. Kelly" wrote:
>
> > Float is an approximate value to begin with.  The gui that is displaying the
> > value may round it off to suite it's needs and I believe that is what you
> > are seeing.  If you want accuracy to that level use decimal and not float.
> >
> > --
> > Andrew J. Kelly  SQL MVP
> >
> >
> > "Phil Cook" <Phil C***@discussions.microsoft.com> wrote in message
> > news:1D95D27B-AD34-4666-9012-704E229BA585@microsoft.com...
> > > When I retrieve a 'float' type value in QA it sometimes displays as
> > > 2768.8400000000001 but when viewing the table in EM it shows as 2768.84,
> > > which I believe to be the correct value.
> > >
> > > What causes this?
> > >
> > >
> > > MSSQL 2000
> > > QA v 8.00.194
> >
> >
> >
Author
18 Jan 2005 3:15 PM
Phil Cook
Many thanks for the explanation Mike.

Show quote
"Mike Epprecht (SQL MVP)" wrote:

> Hi
>
> float is an approximate data type. Not all values can be held correctly.
>
> In your case, 2768.84 gets stored as 2768.8400000000001 as it is not
> possible to represent 2768.8400000000000 in the float data type.
>
> As Andrew said, use Decimal data type if you don't want SQL Server to store
> a 'close enough' value.
>
> Regards
> Mike
>
> "Phil Cook" wrote:
>
> > Thanks Andrew. I could understand if it was a rounding issue, but I'm pretty
> > sure the value was only input with 2 dec places. How can the value have been
> > changed?
> >
> >
> >
> > "Andrew J. Kelly" wrote:
> >
> > > Float is an approximate value to begin with.  The gui that is displaying the
> > > value may round it off to suite it's needs and I believe that is what you
> > > are seeing.  If you want accuracy to that level use decimal and not float.
> > >
> > > --
> > > Andrew J. Kelly  SQL MVP
> > >
> > >
> > > "Phil Cook" <Phil C***@discussions.microsoft.com> wrote in message
> > > news:1D95D27B-AD34-4666-9012-704E229BA585@microsoft.com...
> > > > When I retrieve a 'float' type value in QA it sometimes displays as
> > > > 2768.8400000000001 but when viewing the table in EM it shows as 2768.84,
> > > > which I believe to be the correct value.
> > > >
> > > > What causes this?
> > > >
> > > >
> > > > MSSQL 2000
> > > > QA v 8.00.194
> > >
> > >
> > >

AddThis Social Bookmark Button