144 W.P. Worzel
solutions and, when combined with other local solutions, regional solutions emerge.
Conversely, some solutions would be so dependent on a very limited location, that
solutions that were not highly localized would fail. This suggests that a meta-
analysis of geography to solution effectiveness would help “redraw the map” of
a field of study. For example, identifying microclimates could significantly change
weather predictions for an area.
3.2 Evolutionary Reinforcement Learning (ERL)
Ackley and Littman ( 1991 ) describe a system that combines neural nets and a
genetic algorithm in a simulated world. The idea was to evolve creatures that
could learn from their environment using neural nets to recognize things in the
environment and then act based on the signals produced. To accomplish this,
there are two sets of neural nets: those that respond to sensory input and those
that take action in response to the inputs. The weights in the sensory neural net
are heritable and fixed. The weights used in the action neural net are refined using
a reinforcement learning algorithm during the life of the individual where weights
are reinforced if the environment is evaluated as being better at time t+1 than it was
at time t.
The genetic algorithm is used to create successive generations of creatures,
mostly by crossover of encoded values that are mapped to weights for the neural
nets. Successful individuals in the environment propagate based on the life or death
of the individuals living in the artificial world. Details of the artificial world and
the behaviors engendered by this system are not described here, but the reader is
encouraged to read the details of the dynamics of the system and the population of
individuals in the above cited paper as they are informative and interesting in their
relationship to population dynamics as described in biological literature.
However, the principles of ERL lend themselves to the IoT as well. Here neural
nets would be used to process data from a sensor, and actions would be taken
based on the inputs from the sensors. The results of the actions taken across groups
of sensors would dictate the evolution of sensor behavior. An example might be
weather prediction where the accuracy of prediction based on present data and past
adaptation could lead to more precise forecasts.
3.3 The SKGP
Worzel and MacLean ( 2015 ) described the use of combinators as described in
Turner ( 1979 ) as the basis of a GP system that used a Hindley-Milner type system
(Hindley 1997 ). As detailed in Worzel and MacLean ( 2015 ), it has many virtues,
including first class functions, and natural partitioning of functional components
that work well in a map-reduce environment, and (unique to GP) the fact that