Forum Discussion

FICO's avatar
FICO
Community Manager
7 years ago

What is the code for increasing calculation tolerance?

I am now implementing my program in Xpress Mosel IVE. I have a problem about calculation tolerance in Xpress. I calculate factorial of 20 (20!) by using "prod" function. But i found out that Xpress returns wrong value from factorial of 13 onward. This is because of limited in calculation tolerance in Xpress. I would like to know if there are any commands or codes I should use in order to increase calculation tolerance of a variable? Or any suggestion on how to handle this issue?

fico® xpress

1 Reply

  • FICO's avatar
    FICO
    Community Manager
    7 years ago

    The MAX_INT value of Mosel is 2 to the power of 31 (that is, roughly 2E9). If you need any larger values you should use real numbers, so for example for your case:

      setparam("realfmt", "%.20g") ! Change display format to show up to 20 digits

     forall(n in 10..20) writeln(n, "! = ", prod(i in 1..n) real(i))

    Note that the correctness of the results will depend on what can be achieved / correctly represented in floating point arithmetic on your computer system (Mosel reals are double-precision floating point numbers).

    Mosel tolerance settings do not have any impact on calculations (the parameter 'zerotol' is applied in comparisons between reals).

Related Content

Related Content