Advanced Rails - Building Industrial-Strength Web Apps in Record Time

164 | Chapter 6: Performance

$ perf_comp a/perf_run.searches_create.txt b/perf_run.searches_create.txt

perf data file 1: a/perf_run.searches_create.txt

requests=100, options=-bm=searches_create

perf data file 2: b/perf_run.searches_create.txt

requests=100, options=-bm=searches_create

page c1 real c2 real c1 r/s c2 r/s c1 ms/r c2 ms/r c1/c2

searches_create 44.70490 43.13935 2.2 2.3 447.05 431.39 1.04

all requests 44.70490 43.13935 2.2 2.3 447.05 431.39 1.04

This is mostly information we have seen before. The interesting bit of information is
the far-right number in the table (1.04): the ratio of the old runtime to the new. In
this case, we see that the cache optimization afforded us a 4% improvement in over-
all runtime on this action. This is a decent performance improvement, for an action
that is already fairly well optimized.

Compare that to the second optimization, the refactoring:

$ perf_comp b/perf_run.searches_create.txt c/perf_run.searches_create.txt

perf data file 1: b/perf_run.searches_create.txt

requests=100, options=-bm=searches_create

perf data file 2: c/perf_run.searches_create.txt

requests=100, options=-bm=searches_create

page c1 real c2 real c1 r/s c2 r/s c1 ms/r c2 ms/r c1/c2

searches_create 43.13935 42.79449 2.3 2.3 431.39 427.94 1.01

all requests 43.13935 42.79449 2.3 2.3 431.39 427.94 1.01

This change only resulted in a performance gain of 1%, which is probably not worth
pursuing for performance’s sake alone. (In this case, the code under question was in
desperate need of a rewrite, so this was a net gain anyway.)

We should also compare the actual ranges that the trial times fall into, to be sure that
these results have statistical significance. To visualize this, I instrumented the Rails-
benchPerfInfoclass to show the actual trial times (not just the mean and standard
deviation), and piped the data through R.* The result is shown in Figure 6-5.

This box-and-whisker plot shows the median value (the heavy line), the first and
third quartile (into which the middle half of the data points fall; represented by the
box), and the complete range (the brackets). From this, we see that the first optimi-
zation was a clear winner; the ranges of values do not even overlap.

The second optimization is not as clear-cut; there is still significant overlap between
the two ranges of observations. If there is any performance gain, it is likely marginal,
as it is overshadowed by the inherent variability of the response times.

• R (http://www.r-project.org/) is a language for statistical computing.