SQL set floating point precision

Question

For a SQL int that is being converted to a float, how do I set the precision of the floating point number?

This is the selection I would like to truncate to two or 3 decimal places:

AVG(Cast(e.employee_level as Float))avg_level,

Thanks!

Solution 1

In TSQL, you can specify two different sizes for float, 24 or 53. This will set the precision to 7 or 15 digits respectively.

If all you want to do is truncate to a set number of decimal places, you can use ROUND, ie:

ROUND(AVG(CAST(e.employee_level as float)), 3)

Solution 2

As a general rule, you can't specify the number of digits after the decimal point for a floating-point number. Floating point data types store the closest floating-point approximation to any given value. The closest floating-point approximation is unlikely to have the number of digits you want. Although you might be able to suppress every digit after the third one, that will only change the appearance of the value, not the value itself.

Integers are a different story. An integer--stored, converted, or cast to a floating-point data type--will be stored exactly over a large range. Floating-point data types don't have to store any fractional units for integers.

I'd suggest, though that the best practice for you is to

• avoid casting integers to floating-point if you don't need fractional units, or
• cast integers to decimal or numeric if you do need fractional units, or
• handle display issues entirely in application code.

Comments

[Bort]

Should note that if you use float(n) for some n other than 24 or 53, it gets rounded up to either 24 or 53