Function Precision(x, y)

Precision

Precision(
     x,            ! (input) numerical expression
     y             ! (input) integer expression
     )

Arguments

x

A scalar numerical expression.

y

An integer expression.

Return Value

The function Precision returns x rounded to y significant digits.

Example

_p_val := 123.456789 ;
_p_returnA := precision( _p_val, 1 ); ! precision( _p_val, 1 ) =  100
_p_returnB := precision( _p_val, 3 ); ! precision( _p_val, 3 ) =  123
_p_returnC := precision( _p_val, 5 ); ! precision( _p_val, 5 ) =  123.46
_p_returnD := precision( _p_val, 7 ); ! precision( _p_val, 7 ) =  123.4568

Note

  • The function Precision can be used in constraints of nonlinear mathematical programs. However, nonlinear solvers may experience convergence problems around the discontinuities of the Precision function.

  • When the numerical expression contains a unit, the function Precision will first convert the expression to the corresponding base unit, before evaluating the function itself.

See also

The functions Round, Ceil, Floor, Trunc. Arithmetic functions are discussed in full detail in Arithmetic Functions of the Language Reference.