File Type Partial Loading
Binary You can use low-level binary file I/O
functions, such as fread, to access parts of
any file that has a known format. For binary
files of an unknown format, try using
memory mapping with the memmapfile
function.
Image, HDF, Audio, and Video Many of the MATLAB functions that support
loading from these types of files allow you
to select portions of the data to read. For
details, see the function reference pages
listed in “Supported File Formats for Import
and Export”.Process Data By Blocks
Consider block processing, that is, processing a large data set one section at a time in a
loop. Reducing the size of the largest array in a data set reduces the size of any copies or
temporaries needed. You can use this technique in either of two ways:
- For a subset of applications that you can break into separate chunks and process
independently. - For applications that only rely on the state of a previous block, such as filtering.
Avoid Creating Temporary Arrays
Avoid creating large temporary variables, and also make it a practice to clear those
temporary variables you do use when they are no longer needed. For example, when you
create a large array of zeros, instead of saving to a temporary variable A, and then
converting A to a single:
A = zeros(1e6,1);
As = single(A);
use just the one command to do both operations:
A = zeros(1e6,1,'single');
Using the repmat function, array preallocation and for loops are other ways to work on
nondouble data without requiring temporary storage in memory.
Strategies for Efficient Use of Memory