268 CATALYZING INQUIRY
than any other search algorithm over the space of all problems.^62 Therefore, problem-specific knowl-
edge must be incorporated either implicitly or explicitly in an evolutionary algorithm for it to perform
well. Finally, evolutionary algorithms are dynamical systems, and the systems properties necessary to
make them good search algorithms are well characterized.^63
The primary question that remains to tie together the above is the following: How—and when—can
knowledge about the problem be translated into representations and genetic operators that produce an
evolutionary algorithm with good performance?
In the absence of this critical link in the theory of evolutionary algorithms, the approach taken by
designers resorts to the empirical: try it out and see if it works. Evolutionary approaches provide the
greatest advantage over other methods in cases where it is not understood how to construct answers
from “first principles” (i.e., logico-deductive procedures), but where approximate solutions can be
refined by variation and testing. Such problems can be characterized as “difficult inverse problems,”
where the inverse refers to finding inputs that produce desired outputs of the system in question.
Moreover, evolutionary techniques tend to work best on problems involving relatively large search
spaces and large numbers of variables that are not well understood. Evolutionary algorithms have been
able to construct and adapt complex neural networks that are intractable analytically or for which
derivative-based back-propagation is inapplicable. Genetic programming has produced complex cir-
cuits that infringe on patented inventions. By contrast, problems involving small search spaces can
usually be searched systematically, and search spaces being well understood generally means that
special-purpose heuristics are available. (For example, the Traveling Salesman Problem is reasonably
well understood, and there are very good special heuristics for solving that problem.)
For problems in which evolutionary techniques are unable to find global optima, they may never-
theless find very good approximations that are robust to wide-ranging initial conditions. Thus, the
solutions generated may be adequate to the task at hand. For this reason, evolutionary techniques may
also be better when data are very noisy or in the presence of a varying fitness function: the algorithm
may rapidly produce approximate solutions that track the changing environment, just as evolving
species can track environmental changes. (An example of a problem calling for a varying fitness func-
tion might be a robot that must learn, online, in a dynamic environment, where the task facing the robot
changes over time.)
8.3.1.3 Correctness of a Solution,
One of the most challenging aspects of evolutionary computation is evaluating the correctness of a
solution derived through evolutionary means. Because evolutionary solutions are cumulative, in the
sense that they build on previous solutions, the design process does not have an opportunity to develop
solutions that are clean and elegantly designed from first principles. Human inspection of a solution so
derived is unlikely to yield much insight. Thus, essentially the only way known today to assess the
correctness of such a solution is to subject it to extensive testing. Rather than a human being under-
standing how the solution achieves its goals, the proposed solution convinces a human being that it will
do so.
Note, however, that ascertaining the correctness of any large computational artifact (e.g., a complex
software system or a VLSI chip) depends to a large degree on testing. Of course, because the thought
and decision-making processes of human beings are not available to public inspection, it is only by
observing a human being in action that one develops confidence in the designer’s ability to perform
(^62) D.H. Wolpert and W.G. Macready, “No Free Lunch Theorems for Optimization,” IEEE Transactions on Evolutionary Computa-
tion 1(1):67-82, 1997, available at http://citeseer.ist.psu.edu/wolpert96no.html.
(^63) L. Altenberg, “Open Problems in the Spectral Analysis of Evolutionary Dynamics,” pp. 73-102 in Frontiers of Evolutionary
Computation, A. Menon, ed., Genetic Algorithms and Evolutionary Computation Series, Volume 11, Kluwer Academic Publish-
ers, Boston, MA, 2004.