Exploitation and Exploration

John Holland's principle of balancing '''exploitation and exploration''' (Holland, 1992):
the right balance between exploitation and exploration allows
a [[Complex_Adaptive_System|complex adaptive system]] to focus on the most promising possibilities seen so
far, while looking for new possibilities at the same time as well.
Pure exploitation would not allow the acquisition of new information,
pure exploration would not allow to use available resources efficiently.
Examples are according to Melanie Mitchell the immune system or 
an ant colony which search for intruders and food, respectively: 
when promising possibilities are identified, they should be 
exploited at a rate and intensity related to their estimated 
promise, which is continually updated. But at all times exploration 
for new possibilities should continue.

Therefore the principle is important for [[Adaptation|adaptive]] information processing 
in changing environments. It is also important in many other fields, for
example in reinforcement learning and economics: should companies invest 
in research & development and fund innovative ideas or should they try to 
use their available resources and techniques more efficiently ?
The principle of balancing exploitation and exploration is based
on two core principles of [[Evolution|evolution]]: exploration means
variation (for example by recombination or mutation), while 
natural selection is similar to exploitation.

== Swarm Intelligence ==

Systems which use [[Swarm Intelligence|swarm intelligence]] are often able 
to achieve the right balance between exploration and exploitation.
During food foraging, colonies and swarms must achieve a good trade-off
between exploration (search new sources) and exploitation (use known sources).
Trail-based foraging amont ants and danced-based foraging among honeybees
allow naturally to regulate the number of foragers:
if the trail is weak and the food quality low, many ants will lose the 
connection, and start to explore new areas. The weaker the trail, the 
larger is the number of lost ants who are able to discover new food 
sources.
If the number of dancers in a honeybee colony is low, the unemployed
foragers will leave the colony and start to explore new areas, too.
The lower the number of dances, the larger is the number of scouts which
explore new areas.

== Books ==

* Holland, J. H. 1992. Adaptation in Natural and Artificial Systems. Cambridge, MA: MIT Press. First edition, 1975