This is essentially equivalent to the optimal out-of-core mode for the non-distributed sparse solver.
Therefore, the total memory usage of the distributed sparse solver when using the optimal out-of-core
memory mode is about n times the memory that is needed to hold the largest front. In other words,
as more cores are used the total memory used by the solver (summed across all processes) actually in-
creases when running in this memory mode.
The DSPOPTION command allows you to choose a specific memory strategy for the distributed sparse
solver. The available options for the Memory_Option field are DEFAULT, INCORE, OPTIMAL, and FORCE.
Sophisticated memory usage heuristics, similar to those used by the sparse solver, are used to balance
the specific memory requirements of the distributed sparse solver with the available memory on the
machine(s) being used. By default, most smaller jobs run in the INCORE memory mode, while larger
jobs can run either in the INCORE memory mode or in the OPTIMAL memory mode. In some cases, you
may want to explicitly set the memory mode using the DSPOPTION command. However, this is only
recommended if you fully understand the solver memory used on each machine and the available
memory for each machine.
When the distributed sparse solver runs in the out-of-core memory mode, it does substantial I/O to the
disk storage device on the machine. If multiple solver processes write to the same disk, the performance
of the solver decreases as more solver processes are used, meaning the total elapsed time of the solver
does not decrease as much as expected. The ideal configuration for the distributed sparse solver when
running in out-of-core mode is to run using a single process on each machine in a cluster or network,
spreading the I/O across the hard drives of each machine, assuming that a high-speed network such
as Infiniband is being used. Running the distributed sparse solver in out-of-core mode on a shared disk
resource (for example, NAS or SAN disk) is typically not recommended. You can effectively run the dis-
tributed sparse solver using multiple processes with one drive (or a shared disk resource) if:
- The problem size is small enough relative to the physical memory on the system that the system
buffer cache can hold all of the distributed sparse solver (I/O) files and other files in memory. - You have a very fast hard drive configuration that can handle multiple I/O requests simultaneously.
For a shared disk resource on a cluster, a very fast interconnect is also needed to handle the I/O traffic
along with the regular communication of data within the solver. - You use the DSPOPTION,,INCORE command to force the distributed sparse solver into an in-core
mode.
5.2.2. The Preconditioned Conjugate Gradient (PCG) Solver
The PCG solver starts with element matrix formulation. Instead of factoring the global matrix, the PCG
solver assembles the full global stiffness matrix and calculates the DOF solution by iterating to conver-
gence (starting with an initial guess solution for all DOFs).The PCG solver uses a proprietary precondi-
tioner that is material property and element-dependent.
- The PCG solver is usually about 4 to 10 times faster than the JCG solver for structural solid elements and
about 10 times faster then JCG for shell elements. Savings increase with the problem size. - The PCG solver usually requires approximately twice as much memory as the JCG solver because it retains
two matrices in memory:- The preconditioner, which is almost the same size as the stiffness matrix
- The symmetric, nonzero part of the stiffness matrix
Release 15.0 - © SAS IP, Inc. All rights reserved. - Contains proprietary and confidential informationTypes of Solvers