Warning: calculations with internal math with precision exceeding 10,000 digits are currently impractically slow.
In some algorithms it is necessary to compute the integer parts of expressions such as a*Ln(b)/Ln(10) or a*Ln(10)/Ln(2) where a, b are short integers of order O(P). Such expressions are frequently needed to estimate the number of terms in the Taylor series or similar parameters of the algorithms. In these cases, it is important that the result is not underestimated but it would be wasteful to compute Ln(10)/Ln(2) in floating point only to discard most of that information by taking the integer part of say 1000*Ln(10)/Ln(2). It is more efficient to approximate such constants from above by short rational numbers, for example, Ln(10)/Ln(2)<28738/8651 and Ln(2)<7050/10171. The error of such an approximation will be small enough for practical purposes. The function NearRational can be used to find optimal rational approximations. The function IntLog (see below) efficiently computes the integer part of a logarithm in integer base. If more precision is desired in calculating Ln(a)/Ln(b) for integer a, b, one can compute IntLog(a^k,b) for some integer k and then divide by k.
There are also a function NextPrime(n) that returns the smallest prime number larger than n. This function uses a sequence 5,7,11,13,... generated by the function NextPseudoPrime that contains numbers not divisible by 2 or 3 (but perhaps divisible by 5,7,...). NextPseudoPrime is very fast because it does not test for prime numbers.
For small prime factors the trial division algorithm is used: n is divided by all prime numbers p<=257 until a factor is found. NextPseudoPrime is used to generate the sequence of candidate divisors p.
After separating small prime factors, we test whether the number n is an integer power of a prime number, i.e. whether n=p^s for some prime number p and an integer s>=1. This is tested by the following algorithm. We already know that n is not prime and that n does not contain any small prime factors up to 257. Therefore if n=p^s, then p>257 and 2<=s<s[0]=Ln(n)/Ln(257). In other words, we only need to look for powers not greater than s[0]. This number can be approximated by the "integer logarithm" of n in base 257 (routine IntLog (n, 257)).
Now we need to check whether n is of the form p^s for s=2, 3, ..., s[0]. Note that if for example n=p^24 for some p, then the square root of n will already be an integer, n^(1/2)=p^12. Therefore it is enough to test whether n^(1/s) is an integer for all prime values of s up to s[0], and then we will definitely discover whether n is a power of some other integer. The testing is performed using the integer square root function IntNthRoot which quickly computes the integer part of n-th root of an integer number. If we discover that n has an integer root p of order s, we have to check that p itself is a prime power (we use the same algorithm recursively). The number n is a prime power if and only if p is itself a prime power. If we find no integer roots of orders s<=s[0], then n is not a prime power.
If the number n is not a prime power, the Pollard "rho" algorithm is applied (J. Pollard, Monte Carlo methods for index computation mod p, Mathematics of Computation, volume 32, pages 918-924, 1978). The Pollard "rho" algorithm takes an irreducible polynomial, e.g. p(x)=x^2+1 and builds a sequence of integers x[k+1]:=Mod(p(x[k]),n), starting from x[0]=2. For each k, the value x[2*k]-x[k] is attempted as possibly containing a common factor with n. The GCD of x[2*k]-x[k] with n is computed, and if Gcd(x[2*k]-x[k],n)>1, then that GCD value divides n.
The Pollard "rho" algorithm may enter an infinite loop when the sequence x[k] repeats itself without giving any factors of n. For example, the unmodified "rho" algorithm loops on the number 703. The loop is detected by comparing x[2*k] and x[k]. When these two quantities become equal to each other for the first time, the loop may not yet have occurred so the value of GCD is set to 1 and the sequence is continued. But when the equality of x[2*k] and x[k] occurs many times, it indicates that the algorithm has entered a loop. A solution is to randomly choose a different starting number x[0] when a loop occurs and try factoring again, and keep trying new random starting numbers between 1 and n until a non-looping sequence is found. The current implementation stops after 100 restart attempts and prints an error message, "failed to factorize number".
A better (and faster) integer factoring algorithm is needed.
The algorithm first splits the interval into a specified initial number of equal subintervals, and then repeatedly splits each subinterval in half until the function is well enough approximated by the resulting grid. The integer parameter depth gives the maximum number of binary splittings for a given initial interval; thus, at most 2^depth additional grid points will be generated. The function Plot2D'adaptive should return a list of pairs of points {{x1,y1}, {x2,y2}, ...} to be used directly for plotting.
The recursive bisection algorithm goes like this:
This algorithm works well if the initial number of points and the depth parameter are large enough.
Singularities in the function are handled by the step 3. Namely, the algorithm checks whether the function returns a non-number (e.g. Infinity) and if so, the sign change is always considered to be "too rapid". Thus, the intervals immediately adjacent to the singularity will be plotted at the highest allowed refinement level. When plotting the resulting data, the singular points are simply not printed the data file and the plotting programs do not have any problems.
The meaning of Newton-Cotes quadrature coefficients is that an integral is approximated as
The solution of this system always exists and gives quadrature coefficients as rational numbers. For example, the Simpson quadrature c[0]=1/6, c[1]=2/3, c[2]=1/6 is obtained with n=2.
In the same way it is possible to find quadratures for the integral over a subinterval rather than over the whole interval of x. In the current implementation of the adaptive plotting algorithm, two quadratures are used: the 3-point quadrature ( n=2) and the 4-point quadrature ( n=3) for the integral over the first subinterval, Integrate(x,a[0],a[1])f(x). Their coefficients are (5/12, 2/3, -1/12) and ( 3/8, 19/24, -5/24, 1/24).
The basic formula is widely known: If f(x)=0 must be solved, one starts with a value of x that is close to some root and iterates
Newton's method is particularly convenient for multiple precision calculations because of its insensitivity to accumulated errors: if x[k] at some iteration is found with a small error, the error will be corrected at the next iteration. Therefore it is not necessary to compute all iterations with the full required precision; each iteration needs to be performed at the precision of the root expected from that iteration. For example, if we know that the initial approximation is accurate to 3 digits, then (assuming quadratic convergence) it is enough to perform the first iteration to 6 digits, the second iteration to 12 digits and so on. In this way, multiple precision calculations are enormously speeded up.
However, Newton's method suffers from sensitivity to the initial guess. If the initial value x[0] is not chosen sufficiently close to the root, the iterations may converge very slowly or not converge at all. To remedy this, one can combine Newton's iteration with simple bisection. Once the root is bracketed inside an interval (a, b), one checks whether (a+b)/2 is a better approximation for the root than that obtained from Newton's iteration. This guarantees at least linear convergence in the worst case.
For some equations f(x)=0, Newton's method converges faster; for example, solving Sin(x)=0 in the neighborhood of x=3.14159 gives "cubic" convergence, i.e. the number of correct digits is tripled at each step. This happens because Sin(x) near its root x=Pi has vanishing second derivative and thus the function is particularly well approximated by a straight line.
Halley's method is an improvement to Newton's method that makes each equation well approximated by a straight line near the root. Edmund Halley computed fractional powers, x=a^(1/n), by the iteration
Halley's method can be generalized to any function f(x). A cubically convergent iteration is always obtained if we replace the equation f(x)=0 by an equivalent equation
Halley's iteration, despite its faster convergence rate, may be more cumbersome to evaluate than Newton's iteration and so it may not provide a more efficient numerical method for some functions. Only in some special cases is Halley's iteration just as simple to compute as Newton's iteration. But Halley's method has another advantage: it is generally less sensitive to the choice of the initial point x[0]. An extreme example of sensitivity to the initial point is the equation x^(-2)=12 for which Newton's iteration x'=3*x/2-6*x^3 converges to the root only from initial points 0<x[0]<0.5 and wildly diverges otherwise, while Halley's iteration converges to the root from any x[0]>0.
It is at any rate not true that Halley's method always converges better than Newton's method. For instance, it diverges on the equation 2*Cos(x)=x unless started at x[0] within the interval (-1/6*Pi, 7/6*Pi). Another example is the equation Ln(x)=a. This equation allows to compute x=Exp(a) if a fast method for computing Ln(x) is available (e.g. the AGM-based method). For this equation, Newton's iteration
When it converges, Halley's iteration can still converge very slowly for certain functions f(x), for example, for f(x)=x^n-a if n^n>a. For such functions that have very large and rapidly changing derivatives, no general method can converge faster than linearly. In other words, a simple bisection will generally do just as well as any sophisticated iteration, until the root is approximated relatively precisely. Halley's iteration combined with bisection seems to be a good choice for such problems.
For practical evaluation, iterations must be supplemented with error control. For example, if x0 and x1 are two consecutive approximations that are already very close, we can quickly compute the achieved (relative) precision by finding the number of leading zeros in the number Abs(x0-x1)/Max(x0,x1). This is easily done using the integer logarithm. After performing a small number of initial iterations at low precision, we can make sure that x1 has at least a certain number of correct digits of the root. Then we know which precision to use for the next iteration (e.g. triple precision if we are using a cubically convergent scheme). It is important to perform each iteration at the precision of the root which it will give and not at a higher precision; this saves a great deal of time since multiple-precision calculations quickly become very slow at high precision.
If we do not know in advance how many terms of the Taylor series we need, we cannot do any better than just evaluate each term and check if it is already small enough. So in this case we will have to do O(N) long multiplications. However, we can organize the calculation much more efficiently if we can estimate the necessary number of terms and if we can afford some storage. A "rectangular" algorithm uses 2*Sqrt(N) long multiplications (assuming that the coefficients of the series are short rational numbers) and Sqrt(N) units of storage. (See paper: D. M. Smith, Efficient multiple-precision evaluation of elementary functions, 1985.)
Suppose we need to evaluate Sum(k,0,N,a[k]*x^k) and we know that N terms are enough. Suppose also that the coefficients a[k] are rational numbers with small numerators and denominators, so a multiplication a[k]*x is not a long multiplication (usually, either a[k] or the ratio a[k]/a[k-1] is a short rational number). Then we can organize the calculation in a rectangular array with c columns and r rows like this,
The total required number of long multiplications is r+c+Ln(r)-2. The minimum number of multiplications, given that r*c>=N, is around 2*Sqrt(N) at r<=>Sqrt(N)-1/2 (the formula r<=>Sqrt(N-Sqrt(N)) can be used with an integer square root algorithm). Therefore, by arranging the Taylor series in a rectangle of sides r and c, we have obtained an algorithm which costs O(Sqrt(N)) instead of O(N) long multiplications and requires Sqrt(N) units of storage.
One might wonder if we should not try to arrange the Taylor series in a cube or another multidimensional matrix instead of a rectangle. However, calculations show that this does not save time: the optimal arrangement is the two-dimensional rectangle.
An additional speed-up is possible if the elementary function allows a transformation that reduces x and makes the Taylor series converge faster. For example, Ln(x)=2*Ln(Sqrt(x)), Cos(2*x)=2*Cos(x)^2-1, and Sin(3*x)=3*Sin(x)-4*Sin(x)^3 are such transformations. It may be faster to perform a number of such transformations before evaluating the Taylor series, if the time saved by its quicker convergence is more than the time needed to perform the transformations. The optimal number of transformations can be estimated. Using this technique in principle reduces the cost of Taylor series from O(Sqrt(N)) to O(N^(1/3)) long multiplications. However, additional roundoff error may be introduced by this procedure for some x.
More formally, one can define the (complex) function of two (complex) numbers AGM(x,y) as the limit of the sequence a[k] where a[k+1]=1/2*(a[k]+b[k]), b[k+1]=Sqrt(a[k]*b[k]), and the initial values are a[0]=x, b[0]=y. This function is obviously linear, AGM(k*x,k*y)=k*AGM(x,y), so in principle it is enough to compute AGM(1,x) or arbitrarily select k for convenience.
The limit of the AGM sequence is related to the complete elliptic integral by
The definition of the AGM sequence for complex values requires to take a square root Sqrt(a*b), which needs a branch cut to be well-defined. Selecting the natural cut along the negative real semiaxis (Re(x)<0, Im(x)=0), we obtain a AGM sequence that converges for any initial values x, y with positive real part. If the numbers x and y are very different (one is much larger than another), then the numbers a[k], b[k] become approximately equal after about k=1/Ln(2)*Ln(Abs(Ln(x/y))) iterations (note: Brent's paper mistypes this as 1/Ln(2)*Abs(Ln(x/y))). Then one needs about Ln(n)/Ln(2) more iterations to make the first n decimal digits of a[k] and b[k] coincide, because the relative error epsilon=1-b/a decays approximately as epsilon[k]=1/8*Exp(-2^k).
Unlike the Newton iteration, the AGM sequence does not correct errors and all elements need to be computed with full precision. Actually, slightly more precision is needed to compensate for accumulated roundoff error. Brent (paper rpb028, see below) says that O(Ln(Ln(n))) bits of accuracy are lost to roundoff error if there are total of n iterations.
The AGM sequence can be used for fast computations of Pi, Ln(x) and ArcTan(x). However, currently the limitations of Yacas internal math make these methods less efficient than simpler methods based on Taylor series and Newton iterations.
The square root is computed by the Newton iteration.
A separate function IntNthRoot is provided to compute integer part of n^(1/s) for integer n and s. For a given s, it evaluates the integer part of n^(1/s) using only integer arithmetic with integers of size n^(1+1/s). This can be done by Halley's iteration method, solving the equation x^s=n. For this function, the Halley iteration sequence is monotonic. The initial guess is obtained by bit counting using the integer logarithm function, x[0]=2^(b(n)/s) where b(n) is the number of bits in n. It is clear that the initial guess is accurate to within a factor of 2. Since the relative error is squared at every iteration, we need as many iteration steps as bits in n^(1/s).
Since we only need the integer part of the root, it is enough to use integer division in the Halley iteration. The sequence x[k] will monotonically approximate the number n^(1/s) from below if we start from an initial guess that is less than the exact value. (We start from below so that we have to deal with smaller integers rather than with larger integers.) If n=p^s, then after enough iterations the floating-point value of x[k] would be slightly less than p; our value is the integer part of x[k]. Therefore, at each step we check whether 1+x[k] is a solution of x^s=n, in which case we are done; and we also check whether (1+x[k])^s>n, in which case the integer part of the root is x[k]. To speed up the Halley iteration in the worst case when s^s>n, it is combined with bisection. The root bracket interval x1<x<x2 is maintained and the next iteration x[k+1] is assigned to the midpoint of the interval if the Newton formula does not give sufficiently rapid convergence. The initial root bracket interval can be taken as x[0], 2*x[0].
Real powers (as opposed to integer powers and roots) are computed by using the exponential and logarithm functions, a^b=Exp(b*Ln(a)).
The "integer logarithm", defined as the integer part of Ln(x)/Ln(b), where x and b are integers, is computed using a special routine IntLog(x, b) with purely integer math. This is much faster than evaluating the full logarithm when both arguments are integers and only the integer part of the logarithm is needed. The algorithm consists of (integer) dividing x by b repeatedly until x becomes 0 and counting the number of divisions. A speed-up for large x is achieved by first comparing x with b, then with b^2, b^4, etc., until the factor b^2^n is larger than x. At this point, x is divided by that power of b and the remaining value is iteratively compared with and divided by successively smaller powers of b.
The logarithm function Ln(x) for general (complex) x can be computed using its Taylor series,
Currently the routine LnNum uses the Halley method for the equation Exp(x)=a to find x=Ln(a). This is currently much faster than other methods.
A much faster algorithm based on the AGM sequence was given by Salamin (see R. P. Brent: Multiple-precision zero-finding methods and the complexity of elementary function evaluation, in Analytic Computational Complexity, ed. by J. F. Traub, Academic Press, 1975, p. 151; also available online from Oxford Computing Laboratory, as the paper rpb028). The formula is based on an asymptotic relation,
The required number of AGM iterations is approximately 2*Ln(P)/Ln(2). For smaller values of x (but x>1), one can either raise x to a large integer power s (this is quick only if x is an integer or a rational) and compute 1/r*Ln(x^r), or multiply x by a large integer power of 2 (this is better for floating-point x) and compute Ln(2^s*x)-s*Ln(2). Here the required powers are
If x<1, then (-Ln(1/x)) is computed. Finally, there is a special case when x is very close to 1, where the Taylor series converges quickly but the AGM algorithm requires to multiply x by a large power of 2 and then subtract two almost equal numbers, leading to a great loss of precision. Suppose 1<x<1+10^(-M), where M is large (say of order P). The Taylor series for Ln(1+epsilon) needs about N= -P*Ln(10)/Ln(epsilon)=P/M terms. If we evaluate the Taylor series using the rectangular scheme, we need 2*Sqrt(N) multiplications and Sqrt(N) units of storage. On the other hand, the main slow operation for the AGM sequence is the geometric mean Sqrt(a*b). If Sqrt(a*b) takes an equivalent of c multiplications (Brent's estimate would be c=13/2 but it may be more in practice), then the AGM sequence requires 2*c*Ln(P)/Ln(2) multiplications. Therefore the Taylor series method is more efficient for
For larger x>1+10^(-M), the AGM method is more efficient. It is necessary to increase the working precision to P+M*Ln(2)/Ln(10) but this does not decrease the asymptotic speed of the algorithm. To compute Ln(x) with P digits of precision for any x, only O(Ln(P)) long multiplications are required.
An alternative way to compute x=Exp(a) at large precision would be to solve the equation Ln(x)=a using a fast logarithm routine. A cubically convergent formula is obtained if we replace Ln(x)=a by an equivalent equation
Inverse trigonometric functions are computed by Newton's method (for ArcSin) or by continued fraction expansion (for ArcTan),
By the identity ArcCos(x):=Pi/2-ArcSin(x), the inverse cosine is reduced to the inverse sine. Newton's method for ArcSin(x) consists of solving the equation Sin(y)=x for y. Implementation is similar to the calculation of pi in PiMethod0().
For x close to 1, Newton's method for ArcSin(x) converges very slowly. An identity
Inverse tangent can also be related to inverse sine by
Hyperbolic and inverse hyperbolic functions are reduced to exponentials and logarithms: Cosh(x)=1/2*(Exp(x)+Exp(-x)), Sinh(x)=1/2*(Exp(x)-Exp(-x)), Tanh(x)=Sinh(x)/Cosh(x),
The idea to use continued fraction expansions for ArcTan comes from the book by Jack W. Crenshaw, MATH Toolkit for REAL-TIME Programming (CMP Media Inc., 2000). In that book the author explains how he got the idea to use continued fraction expansions to approximate ArcTan(x), given that the Taylor series converges slowly, and having a hunch that in that case the continued fraction expansion then converges rapidly. He then proceeds to show that in the case of ArcTan(x), this is true in a big way. Now, it might not be true for all slowly converging series. No articles or books have been found yet that prove this. The above book shows it empirically.
One disadvantage of both continued fraction expansions and approximation by rational functions, compared to a simple series, is that it is in general not easy to do the calculation with one step more precision, due to the nature of the form of the expressions, and the way in which they change when expressions with one order better precision are considered. The coefficients of the terms in the polynomials defining the numerator and the denominator of the rational function change. This contrasts with a Taylor series expansion, where each additional term improves the accuracy of the result, and the calculation can be terminated when sufficient accuracy is achieved.
The convergence of the continued fraction expansion of ArcTan(x) is indeed better than convergence of the Taylor series. Namely, the Taylor series converges only for Abs(x)<1 while the continued fraction converges for all x. However, the speed of its convergence is not uniform in x; the larger the value of x, the slower the convergence. The necessary number of terms of the continued fraction is in any case proportional to the required number of digits of precision, but the constant of proportionality depends on x.
This can be understood by the following elementary argument. The difference between two partial continued fractions that differ only by one extra last term can be estimated by
Efficient iterative algorithms for computing pi with arbitrary precision have been recently developed by Brent, Salamin, Borwein and others. However, limitations of the current multiple-precision implementation in Yacas (compiled with the "internal" math option) make these advanced algorithms run slower because they require many more arbitrary-precision multiplications at each iteration.
The file examples/pi.ys implements five different algorithms that duplicate the functionality of Pi(). See http://numbers.computation.free.fr/Constants/ for details of computations of pi and generalizations of Newton-Raphson iteration.
PiMethod0(), PiMethod1(), PiMethod2() are all based on a generalized Newton-Raphson method of solving equations.
Since pi is a solution of Sin(x)=0, one may start sufficiently close, e.g. at x0=3.14159265 and iterate x'=x-Tan(x). In fact it is faster to iterate x'=x+Sin(x) which solves a different equation for pi. PiMethod0() is the straightforward implementation of the latter iteration. A significant speed improvement is achieved by doing calculations at each iteration only with the precision of the root that we expect to get from that iteration. Any imprecision introduced by round-off will be automatically corrected at the next iteration.
If at some iteration x=pi+epsilon for small epsilon, then from the Taylor expansion of Sin(x) it follows that the value x' at the next iteration will differ from pi by O(epsilon^3). Therefore, the number of correct digits triples at each iteration. If we know the number of correct digits of pi in the initial approximation, we can decide in advance how many iterations to compute and what precision to use at each iteration.
The final speed-up in PiMethod0() is to avoid computing at unnecessarily high precision. This may happen if, for example, we need to evaluate 200 digits of pi starting with 20 correct digits. After 2 iterations we would be calculating with 180 digits; the next iteration would have given us 540 digits but we only need 200, so the third iteration would be wasteful. This can be avoided by first computing pi to just over 1/3 of the required precision, i.e. to 67 digits, and then executing the last iteration at full 200 digits. There is still a wasteful step when we would go from 60 digits to 67, but much less time would be wasted than in the calculation with 200 digits of precision.
Newton's method is based on approximating the function f(x) by a straight line. One can achieve better approximation and therefore faster convergence to the root if one approximates the function with a polynomial curve of higher order. The routine PiMethod1() uses the iteration
Both PiMethod0() and PiMethod1() require a computation of Sin(x) at every iteration. An industrial-strength arbitrary precision library such as gmp can multiply numbers much faster than it can evaluate a trigonometric function. Therefore, it would be good to have a method which does not require trigonometrics. PiMethod2() is a simple attempt to remedy the problem. It computes the Taylor series for ArcTan(x),
The routines PiBrentSalamin() and PiBorwein() are based on much more advanced mathematics. (See papers of P. Borwein for review and explanations of the methods.) They do not require evaluations of trigonometric functions, but they do require taking a few square roots at each iteration, and all calculations must be done using full precision. Using modern algorithms, one can compute a square root roughly in the same time as a division; but Yacas's internal math is not yet up to it. Therefore, these two routines perform poorly compared to the more simple-minded PiMethod0().
The algorithm for converting a rational number r=n/m into a continued fraction is simple. First, we determine the integer part of r, which is Div(n,m). If it is negative, we need to subtract one, so that r=n[0]+x and the remainder x is nonnegative and less than 1. The remainder x=Mod(n,m)/m is then inverted, r[1]:=1/x=m/Mod(n,m) and so we have completed the first step in the decomposition, r=n[0]+1/r[1]; now n[0] is integer but r[1] is perhaps not integer. We repeat the same procedure on r[1], obtain the next integer term n[1] and the remainder r[2] and so on, until such n that r[n] is an integer and there is no more work to do. This process will always terminate because all floating-point values are actually rationals in disguise.
Continued fractions are useful in many ways. For example, if we know that a certain number x is rational but have only a floating-point representation of x with a limited precision, say, 1.5662650602409638, we can try to guess its rational form (in this example x=130/83). The function GuessRational uses continued fractions to find a rational number with "optimal" (small) numerator and denominator that is approximately equal to a given floating-point number.
Consider the following example. The number 17/3 has a continued fraction expansion {5,1,2}. Evaluated as a floating point number with limited precision, it may become something like 17/3+0.00001, where the small number represents a roundoff error. The continued fraction expansion of this number is {5, 1, 2, 11110, 1, 5, 1, 3, 2777, 2}. The presence of an unnaturally large term 11110 clearly signifies the place where the floating-point error was introduced; all terms following it should be discarted to recover the continued fraction {5,1,2} and from it the initial number 17/3.
If a continued fraction for a number x is cut right before an unusually large term, and evaluated, the resulting rational number is very close to close to x but has an unusually small denominator. This works because partial continued fractions provide "optimal" rational approximations for the final (irrational) number, and because the magnitude of the terms of the partial fraction is related to the magnitude of the denominator of the resulting rational approximation.
GuessRational(x, prec) needs to choose the place where it should cut the continued fraction. The algorithm for this is somewhat imprecise but works well enough. We can try to find an upper bound for the difference of continued fractions that differ only by an additional last term,
The above estimate for delta hinges on the inequality
This algorithm works well if x is computed with enough precision; namely, it must be computed to at least as many digits as there are in the numerator and the denominator of the fraction combined. Also, the parameter prec should not be too large (or else the algorithm will find a rational number with a larger denominator that approximates x even better).
The related function NearRational(x, prec) works somewhat differently. The goal is to find an "optimal" rational number, i.e. with smallest numerator and denominator, that is within the distance 10^(-prec) of a given value x. The algorithm for this comes from the 1972 HAKMEM document, Item 101C. Their description is terse but clear:
Problem: Given an interval, find in it the rational number with the smallest numerator and denominator. Solution: Express the endpoints as continued fractions. Find the first term where they differ and add 1 to the lesser term, unless it's last. Discard the terms to the right. What's left is the continued fraction for the "smallest" rational in the interval. (If one fraction terminates but matches the other as far as it goes, append an infinity and proceed as above.) |
The HAKMEM text (M. Beeler, R. W. Gosper, and R. Schroeppel: Memo No. 239, MIT AI Lab, 1972, available as HTML online from various places) contains several interesting insights relevant to continued fractions and other numerical algorithms.
There are two tasks related to the factorial: the exact integer calculation and an approximate calculation to some floating-point precision. Factorial of n has approximately n*Ln(n)/Ln(10) decimal digits, so an exact calculation is practical only for relatively small n. In the current implementation, exact factorials for n>65535 are not computed but print an error message advising the user to avoid exact computations. For example, LnGammaNum(n+1) is able to compute Ln(n!) for very large n to the desired floating-point precision.
A second method uses a binary tree arrangement of the numbers 1, 2, ..., n similar to the recursive sorting routine ("merge-sort"). If we denote by a *** b the "partial factorial" product a*(a+1)*...(b-1)*b, then the tree-factorial algorithm consists of replacing n! by 1***n and recursively evaluating (1***m)*((m+1)***n) for some integer m near n/2. The partial factorials of nearby numbers such as m***(m+2) are evaluated explicitly. The binary tree algorithm requires one multiplication of P/2 digit integers at the last step, two P/4 digit multiplications at the last-but-one step and so on. There are O(Ln(n)) total steps of the recursion. If the cost of multiplication is M(P)=P^(1+a)*Ln(P)^b, then one can show that the total cost of the binary tree algorithm is O(M(P)) if a>0 and O(M(P)*Ln(n)) if a=0 (which is the best asymptotic multiplication algorithm).
Therefore, the tree method wins over the simple method if the cost of multiplication is lower than quadratic.
The tree method can also be used to compute "staggered factorials" ( n!!). This is faster than to use the identities (2*n)!! =2^n*n! and
Binomial coefficients Bin(n,m) are found by first selecting the smaller of m, n-m and using the identity Bin(n,m)=Bin(n,n-m). Then a partial factorial is used to compute Bin(n,m)=((n-m+1)***n)/m!. This is always much faster than computing the three factorials in the definition of Bin(n,m).
For real integers n>0, the Gamma function is the same as the factorial,
The Gamma function is implemented as Gamma(x). At integer values n of the argument, Gamma(n) is computed exactly. For half-integer values it is also computed exactly, using the following identities (here n is a nonnegative integer):
For arbitrary complex arguments with nonnegative real part, the library function GammaNum(x) computes a uniform appoximation of Lanczos and Spouge (with the so-called "less precise coefficients of Spouge"). See: C. J. Lanczos, J. SIAM of Num. Anal. Ser. B, vol. 1, 86 (1964); J. L. Spouge, J. SIAM of Num. Anal., vol. 31, 931 (1994). See also: Paul Godfrey 2001 (unpublished): http://winnie.fit.edu/~gabdo/gamma.txt for some explanations on the method. The method gives the Gamma-function only for arguments with positive real part; at negative values of the real part of the argument, the Gamma-function is computed via the identity
The approximation formula used in Yacas depends on a parameter a,
The coefficients c[k] and the parameter a can be chosen to achieve a greater precision of the approximation formula. However, the recipe for the coefficients c[k] given in the paper by Lanczos is too complicated for practical calculations in arbitrary precision: the time it would take to compute the array of N coefficients c[k] grows as N^3. Therefore it is better to use less precise but much simpler formulae derived by Spouge.
In the calculation of the sum Sum(k,1,N,c[k]*(z+k)^(-1)), round-off error can lead to a serious loss of precision. At version 1.0.49, Yacas is limited in its internal arbitrary precision facility in that does not support floating-point numbers with mantissa; this hinders precise calculations with floating-point numbers. (This concern does not apply to Yacas linked with gmp.) In the current version of the GammaNum() function, two workarounds are implemented. First, a Horner scheme is used to compute the sum; this is somewhat faster and leads to smaller roundoff errors. Second, intermediate calculations are performed at 40% higher precision than requested. This is much slower but allows to obtain results at desired precision.
If the factorial of a large integer or half-integer n needs to be computed not exactly but only with a certain floating-point precision, it is faster (for large enough Abs(n)) not to evaluate an exact integer product, but to use the GammaNum approximation or Stirling's asymptotic formula,
The classic book of Bateman and Erdelyi, Higher Transcendental Functions, vol. 1, describes many results concerning analytic properties of zeta(s).
For the numerical evaluation of Riemann's Zeta function with arbitrary precision to become feasible, one needs special algorithms. Recently P. Borwein gave a simple and quick approximation algorithm for positive Re(s) (P. Borwein, An efficient algorithm for Riemann Zeta function (1995), published online and in Canadian Math. Soc. Conf. Proc., 27 (2000), 29-34.) See also: J. M. Borwein, D. M. Bradley, R. E. Crandall: Computation strategies for the Riemann Zeta function, preprint CECM-98-118, 1999, for a review of methods.
It is the "third" algorithm (the simplest one) from P. Borwein's paper which is implemented in Yacas. The approximation formula valid for Re(s)> -(n-1) is
The function Zeta(s) calls ZetaNum(s) to compute this approximation formula for Re(s)>1/2 and uses the identity above to get the value for other s.
For very large values of s, it is faster to use another method implemented in the routine ZetaNum1(s, N). If the required precision is P digits and s>1+Ln(10)/Ln(P)*P, then it is enough to compute the defining series for zeta(n),
For large values of Abs(x), there is the following asymptotic series:
The error of a truncated asymptotic series is not larger than the first discarded term if the number of terms is larger than n-1/2. (See the book: F. W. J. Olver, Asymptotic and special functions, Academic Press, 1974.)
Currently Yacas can compute BesselJ(n,x) for all x where n is an integer and for Abs(x)<=2*Gamma(n) when n is a real number. Yacas currently uses the Taylor series when Abs(x)<=2*Gamma(n) to compute the numerical value:
If Abs(x)>2*Gamma(n) and n is an integer, then Yacas uses the forward recursion identity:
We see from the definition that when Abs(x)<=2*Gamma(n), the absolute value of each term is always decreasing (which is called absolutely monotonely decreasing.) From this we know that if we stop after i iterations, the error will be bounded by the absolute value of the next term. So given a set precision, turn this into a value epsilon, so that we can check if the current term will contribute to the sum at the prescribed precision. Before doing this, Yacas currently increases the precision by 20% to do interim calculations. This is a heuristic that works, it is not backed by theory. The value epsilon is given by epsilon:=5*10^(-prec), where prec was the previous precision. This is directly from the definition of floating point number which is correct to prec digits: A number correct to prec digits has a rounding error no greater than 5*10^(-prec). Beware that some books incorrectly have .5 instead of 5.
Bug: Something is not right with complex numbers, but pure imaginary are OK.
Bernoulli numbers and polynomials are used in various Taylor series expansions, in the Euler-Maclauren series resummation formula, in Riemann's Zeta function and so on. For example, the sum of (integer) p-th powers of consecutive integers is given by
The Bernoulli polynomials B(x)[n] can be found by first computing an array of Bernoulli numbers up to B[n] and then applying the above formula for the coefficients.
In this definition, the first Bernoulli numbers are B[0]=1, B[1]= -1/2, B[2]=4, B[3]=0, B[4]= -1/30.
We consider two distinct computational tasks: evaluate a Bernoulli number exactly as a rational, or find it approximately to a specified floating-point precision. There are also two possible problem settings: either we need to evaluate all Bernoulli numbers B[n] up to some n, or we only need one isolated value B[n] for some n. Depending on how large n is, different algorithms need to be used in these cases.
Here is an estimate of the cost of BernoullliArray. Suppose P is the required number of digits and M(P) is the time to multiply P-digit integers. The required number of digits P to store the numerator of B[n] is asymptotically P<>n*Ln(n). At each of the n iterations we need to multiply O(n) large rational numbers by large coefficients and take a GCD to simplify the resulting fractions. The time for GCD is logarithmic in P. So the complexity of this algorithm is O(n^2*M(P)*Ln(P)) with P<>n*Ln(n).
For large (even) values of the index n, the Bernoulli numbers B[n] are computed by a more efficient procedure: the integer part and the fractional part of B[n] are found separately.
First, by the theorem of Clausen -- von Staudt, the fractional part of (-B[n]) is the same as the fractional part of the sum of all inverse prime numbers p such that n is divisible by p-1. To illustrate the theorem, take n=10 with B[10]=5/66. The number n=10 is divisible only by 1, 2, 5, and 10; this corresponds to p=2, 3, 6 and 11. Of these, 6 is not a prime. Therefore, we exclude 6 and take the sum 1/2+1/3+1/11=61/66. The theorem now says that 61/66 has the same fractional part as -B[10]; in other words, -B[10]=i+f where i is some unknown integer and the fractional part f is a nonnegative rational number, 0<=f<1, which is now known to be 61/66. Indeed -B[10]= -1+61/66. So one can find the fractional part of the Bernoulli number relatively quickly by just checking the numbers that might divide n.
Now one needs to obtain the integer part of B[n]. The number B[n] is positive if Mod(n,4)=2 and negative if Mod(n,4)=0. One can use Riemann's Zeta function identity for even integer values of the argument and compute the value zeta(n) precisely enough so that the integer part of the Bernoulli number is determined. The required precision is found by estimating the Bernoulli number using the same identity in which one approximates zeta(n)=1 and uses Stirling's asymptotic formula for the factorial of large numbers,
At such large values of the argument n, it is feasible to compute the defining series for zeta(n),
Alternatively, one can use the infinite product over prime numbers p[i]
For example, let us compute B[20] using this method.
In> 1/2 + 1/3 + 1/5 + 1/11; Out> 371/330; |
In> N(1+1/2^20) Out> 1.0000009536; |
In> N( 2*20! /(2*Pi)^20*1.0000009536 ) Out> 529.1242423667; |
In> -(529+41/330); Out> -174611/330; |
All these steps are implemented in the routine Bernoulli1. The variable Bernoulli1Threshold determines the smallest n for which B[n] is to be computed via this routine instead of the recursion relation. Its current value is 20.
The complexity of Bernoulli1 is estimated as the complexity of finding all primes up to n plus the complexity of computing the factorial, the power and the Zeta function. Finding the prime numbers up to n by checking all potential divisors up to Sqrt(n) requires O(n^(3/2)*M(Ln(n))) operations with precision O(Ln(n)) digits. In the second step we need to evaluate n!, Pi^n and zeta(n) with precision of P=O(n*Ln(n)) digits. The factorial is found in n short multiplications with P-digit numbers (giving O(n*P)), the power of pi in Ln(n) long multiplications (giving O(Ln(n)*M(P))), and ZetaNum2(n) (the asymptotically faster algorithm) requires O(n*M(P)) operations. The Zeta function calculation dominates the total cost because M(P) is more costly than O(P). So the total complexity of Bernoulli1 is O(n*M(P)) with P<>n*Ln(n).
Note that this is the cost of finding just one Bernoulli number, as opposed to the O(n^2*M(P)*Ln(P)) cost of finding all Bernoulli numbers up to B[n] using the first algorithm BernoulliArray. If we need a complete table of Bernoulli numbers, then BernoulliArray is only marginally (logarithmically) slower. So for finding complete Bernoulli tables, Bernoulli1 is better only for very large n.
However, the recurrence relation used in BernoulliArray turns out to be numerically unstable and needs to be replaced by another (R. P. Brent, "A FORTRAN multiple-precision arithmetic package", ACM TOMS vol. 4, no. 1 (1978), p. 57). Brent's algorithm computes the Bernoulli numbers divided by factorials, C[n]:=B[2*n]/(2*n)! using a (numerically stable) recurrence relation
The numerical instability of the usual recurrence relation
The eigenvalue of the sequence e[k] can be found approximately for large k if we notice that the recurrence relation for e[k] is similar to the truncated Taylor series for Sin(x). Substituting e[k]=lambda^k into it and disregarding a very small number (2*Pi)^(-2*k) on the right hand side, we find
By a very similar calculation one finds that the inverse powers of 4 in Brent's recurrence make the largest eigenvalue of the error sequence e[k] almost equal to 1 and therefore the recurrence is stable. Brent gives the relative error in the computed C[k] as O(k^2) times the roundoff error in the last digit of precision.
The complexity of Brent's method is given as O(n^2*P+n*M(P)) for finding all Bernoulli numbers up to B[n] with precision P digits. This computation time can be achieved if we compute the inverse factorials and powers of 4 approximately by floating-point routines that know how much precision is needed for each term in the recurrence relation. The final long multiplication by (2*k)! computed to precision P adds M(P) to each Bernoulli number.
The non-iterative method using the Zeta function does not perform much better if a Bernoulli number B[n] has to be computed with significantly fewer digits P than the full O(n*Ln(n)) digits needed to represent the integer part of B[n]. (The fractional part of B[n] can always be computed relatively quickly.) The Zeta function needs 10^(P/n) terms, so its complexity is O(10^(P/n)*M(P)) (here by assumption P is not very large so 10^(P/n)<n/(2*Pi*e); if n>P we can disregard the power of 10 in the complexity formula). We should also add O(Ln(n)*M(P)) needed to compute the power of 2*Pi. The total complexity of Bernoulli1 is therefore O(Ln(n)*M(P)+10^(P/n)*M(P)).
If only one Bernoulli number is required, then Bernoulli1 is always faster. If all Bernoulli numbers up to a given n are required, then Brent's recurrence is faster for certain (small enough) n.
Currently Brent's recurrence is implemented as BernoulliArray1() but it is not used by Bernoulli because the internal arithmetic is not yet able to correctly compute with floating-point precision.