MATLAB Programming Fundamentals - MathWorks

Run Performance Tests

There are two ways to run performance tests:

• Use the runperf function to run the tests. This function uses a variable number of
  measurements to reach a sample mean with a 0.05 relative margin of error within a
  0.95 confidence level. It runs the tests four times to warm up the code and between 4
  and 256 times to collect measurements that meet the statistical objectives.


• Generate an explicit test suite using the testsuite function or the methods in the
  TestSuite class, and then create and run a time experiment.
  
  
  • Use the withFixedSampleSize method of the TimeExperiment class to
    construct a time experiment with a fixed number of measurements. You can specify
    a fixed number of warm-up measurements and a fixed number of samples.
  
  
  • Use the limitingSamplingError method of the TimeExperiment class to
    construct a time experiment with specified statistical objectives, such as margin of
    error and confidence level. Also, you can specify the number of warm-up
    measurements and the minimum and maximum number of samples.
  
  
  
  

You can run your performance tests as regression tests. For more information, see “Run
Tests for Various Workflows” on page 33-105.

Understand Invalid Test Results

In some situations, the MeasurementResult for a test result is marked invalid. A test
result is marked invalid when the performance testing framework sets the Valid
property of the MeasurementResult to false. This invalidation occurs if your test fails or
is filtered. Also, if your test incorrectly uses the startMeasuring and stopMeasuring
methods of matlab.perftest.TestCase, then the MeasurementResult for that test
is marked invalid.

When the performance testing framework encounters an invalid test result, it behaves
differently depending on the type of time experiment:

• If you create a frequentist time experiment, then the framework stops measuring for
  that test and moves to the next test.


• If you create a fixed time experiment, then the framework continues collecting the
  specified number of samples.

Overview of Performance Testing Framework