|
dev
newsgroups
|
|||||||||||||||||||||||
|
|||||||||||||||||||||||
Query Analyzer and EM display different resultsWhen I retrieve a 'float' type value in QA it sometimes displays as
2768.8400000000001 but when viewing the table in EM it shows as 2768.84, which I believe to be the correct value. What causes this? MSSQL 2000 QA v 8.00.194 Float is an approximate value to begin with. The gui that is displaying the
value may round it off to suite it's needs and I believe that is what you are seeing. If you want accuracy to that level use decimal and not float. -- Show quoteAndrew J. Kelly SQL MVP "Phil Cook" <Phil C***@discussions.microsoft.com> wrote in message news:1D95D27B-AD34-4666-9012-704E229BA585@microsoft.com... > When I retrieve a 'float' type value in QA it sometimes displays as > 2768.8400000000001 but when viewing the table in EM it shows as 2768.84, > which I believe to be the correct value. > > What causes this? > > > MSSQL 2000 > QA v 8.00.194 Thanks Andrew. I could understand if it was a rounding issue, but I'm pretty
sure the value was only input with 2 dec places. How can the value have been changed? Show quote "Andrew J. Kelly" wrote: > Float is an approximate value to begin with. The gui that is displaying the > value may round it off to suite it's needs and I believe that is what you > are seeing. If you want accuracy to that level use decimal and not float. > > -- > Andrew J. Kelly SQL MVP > > > "Phil Cook" <Phil C***@discussions.microsoft.com> wrote in message > news:1D95D27B-AD34-4666-9012-704E229BA585@microsoft.com... > > When I retrieve a 'float' type value in QA it sometimes displays as > > 2768.8400000000001 but when viewing the table in EM it shows as 2768.84, > > which I believe to be the correct value. > > > > What causes this? > > > > > > MSSQL 2000 > > QA v 8.00.194 > > > Hi
float is an approximate data type. Not all values can be held correctly. In your case, 2768.84 gets stored as 2768.8400000000001 as it is not possible to represent 2768.8400000000000 in the float data type. As Andrew said, use Decimal data type if you don't want SQL Server to store a 'close enough' value. Regards Mike Show quote "Phil Cook" wrote: > Thanks Andrew. I could understand if it was a rounding issue, but I'm pretty > sure the value was only input with 2 dec places. How can the value have been > changed? > > > > "Andrew J. Kelly" wrote: > > > Float is an approximate value to begin with. The gui that is displaying the > > value may round it off to suite it's needs and I believe that is what you > > are seeing. If you want accuracy to that level use decimal and not float. > > > > -- > > Andrew J. Kelly SQL MVP > > > > > > "Phil Cook" <Phil C***@discussions.microsoft.com> wrote in message > > news:1D95D27B-AD34-4666-9012-704E229BA585@microsoft.com... > > > When I retrieve a 'float' type value in QA it sometimes displays as > > > 2768.8400000000001 but when viewing the table in EM it shows as 2768.84, > > > which I believe to be the correct value. > > > > > > What causes this? > > > > > > > > > MSSQL 2000 > > > QA v 8.00.194 > > > > > > Many thanks for the explanation Mike.
Show quote "Mike Epprecht (SQL MVP)" wrote: > Hi > > float is an approximate data type. Not all values can be held correctly. > > In your case, 2768.84 gets stored as 2768.8400000000001 as it is not > possible to represent 2768.8400000000000 in the float data type. > > As Andrew said, use Decimal data type if you don't want SQL Server to store > a 'close enough' value. > > Regards > Mike > > "Phil Cook" wrote: > > > Thanks Andrew. I could understand if it was a rounding issue, but I'm pretty > > sure the value was only input with 2 dec places. How can the value have been > > changed? > > > > > > > > "Andrew J. Kelly" wrote: > > > > > Float is an approximate value to begin with. The gui that is displaying the > > > value may round it off to suite it's needs and I believe that is what you > > > are seeing. If you want accuracy to that level use decimal and not float. > > > > > > -- > > > Andrew J. Kelly SQL MVP > > > > > > > > > "Phil Cook" <Phil C***@discussions.microsoft.com> wrote in message > > > news:1D95D27B-AD34-4666-9012-704E229BA585@microsoft.com... > > > > When I retrieve a 'float' type value in QA it sometimes displays as > > > > 2768.8400000000001 but when viewing the table in EM it shows as 2768.84, > > > > which I believe to be the correct value. > > > > > > > > What causes this? > > > > > > > > > > > > MSSQL 2000 > > > > QA v 8.00.194 > > > > > > > > > |
|||||||||||||||||||||||