factorization. However, convergence of iterative methods is not guaranteed and the number of iterations
required to reach an acceptable solution may be so large that direct methods are faster in some cases.
Because the sparse direct solver is based on direct elimination, poorly conditioned matrices do not pose
any difficulty in producing a solution (although accuracy may be compromised). Direct factorization
methods always give an answer if the equation system is not singular. When the system is close to
singular, the solver can usually give a solution (although you must verify the accuracy).
The sparse solver can run completely in memory (also known as in-core) if sufficient memory is available.
The sparse solver can also run efficiently by using a balance of memory and disk usage (also known as
out-of-core). The out-of-core mode typically requires about the same memory usage as the PCG solver
(~1 GB per million DOFs) and requires a large disk file to store the factorized matrix (~10 GB per million
DOFs). The amount of I/O required for a typical static analysis is three times the size of the matrix fac-
torization. Running the solver factorization in-core (completely in memory) for modal/buckling runs can
save significant amounts of wall (elapsed) time because modal/buckling analyses require several factor-
izations (typically 2 - 4) and repeated forward/backward substitutions (10 - 40+ block solves are typical).
The same effect can often be seen with nonlinear or transient runs which also have repeated factor/solve
steps.
The BCSOPTION command allows you to choose a memory strategy for the sparse solver. The available
options for the Memory_Option field are DEFAULT, INCORE, OPTIMAL, MINIMUM, and FORCE. Depending
on the availability of memory on the system, each memory strategy has its benefits. For systems with
a large amount of physical memory, the INCORE memory mode often results in the best performance.
Conversely, the MINIMUM memory mode often gives the worst solver performance and, therefore, is
only recommended if the other memory options do not work due to limited memory resources. In most
cases you should use the DEFAULT memory mode. In this mode, the sparse solver uses sophisticated
memory usage heuristics to balance available memory with the specific memory requirements of the
sparse solver for each job. By default, most smaller jobs automatically run in the INCORE memory mode,
but larger jobs may run in the INCORE memory mode or in the OPTIMAL memory mode. In some cases
you may want to explicitly set the sparse solver memory mode or memory allocation size using the
BCSOPTION command. However, doing so is only recommended if you know how much physical
memory is on the system and understand the sparse solver memory requirements for the job in question.
When the sparse solver is selected in Distributed ANSYS, the distributed sparse direct solver is automat-
ically used instead. The distributed sparse solver is mathematically identical to the shared-memory
parallel sparse solver and is insensitive to ill-conditioning. It should typically be used for problems with
which the PCG and JCG have convergence difficulty and on computer systems where large memory is
available.
5.2.1.1. Distributed Sparse Direct Solver
The distributed sparse direct solver decomposes a large sparse matrix into smaller submatrices (instead
of decomposing element domains), and then sends these submatrices to multiple cores on either shared-
memory (e.g., server) or distributed-memory (e.g., cluster) hardware. To use more than two cores with
this solver, you must have additional HPC licenses for each core beyond the first two. (For more inform-
ation, see HPC Licensing in the Parallel Processing Guide.)
During the matrix factorization phase, each distributed process factorizes its submatrices simultaneously
and communicates the information as necessary. The submatrices are automatically split into pieces
(or fronts) by the solver during the factorization step. The non-distributed sparse solver works on one
front at a time, while the distributed sparse solver works on n fronts at the same time (where n is the
total number of processes used). Each front in the distributed sparse solver is stored in-core by each
process while it is factored, even while the distributed sparse solver is running in out-of-core mode.
Release 15.0 - © SAS IP, Inc. All rights reserved. - Contains proprietary and confidential informationSolution