Exploitation and Exploration: Difference between revisions
No edit summary |
|||
| (8 intermediate revisions by 2 users not shown) | |||
| Line 12: | Line 12: | ||
for new possibilities should continue. | for new possibilities should continue. | ||
Therefore the principle is important for adaptive information processing | Therefore the principle is important for [[Adaptation|adaptive]] information processing | ||
in changing environments. It is also important in many other fields, for | in changing environments. It is also important in many other fields, for | ||
example in reinforcement learning and economics: should companies invest | example in reinforcement learning and economics: should companies invest | ||
in | in research & development and fund innovative ideas or should they try to | ||
and techniques more efficiently ? | use their available resources and techniques more efficiently ? | ||
The principle of balancing exploitation and exploration is based | |||
on two core principles of [[Evolution|evolution]]: exploration means | |||
variation (for example by recombination or mutation), while | |||
natural selection is similar to exploitation. | |||
== Swarm Intelligence == | |||
Systems which use [[Swarm Intelligence|swarm intelligence]] are often able | |||
to achieve the right balance between exploration and exploitation. | |||
During food foraging, colonies and swarms must achieve a good trade-off | |||
between exploration (search new sources) and exploitation (use known sources). | |||
Trail-based foraging amont ants and danced-based foraging among honeybees | |||
allow naturally to regulate the number of foragers: | |||
if the trail is weak and the food quality low, many ants will lose the | |||
connection, and start to explore new areas. The weaker the trail, the | |||
larger is the number of lost ants who are able to discover new food | |||
sources. | |||
If the number of dancers in a honeybee colony is low, the unemployed | |||
foragers will leave the colony and start to explore new areas, too. | |||
The lower the number of dances, the larger is the number of scouts which | |||
explore new areas. | |||
== Books == | == Books == | ||
* Holland, J. H. 1992. Adaptation in Natural and Artificial Systems. Cambridge, MA: MIT Press. First edition, 1975 | * Holland, J. H. 1992. Adaptation in Natural and Artificial Systems. Cambridge, MA: MIT Press. First edition, 1975 | ||
Latest revision as of 16:40, 11 February 2011
John Holland's principle of balancing exploitation and exploration (Holland, 1992): the right balance between exploitation and exploration allows a complex adaptive system to focus on the most promising possibilities seen so far, while looking for new possibilities at the same time as well. Pure exploitation would not allow the acquisition of new information, pure exploration would not allow to use available resources efficiently. Examples are according to Melanie Mitchell the immune system or an ant colony which search for intruders and food, respectively: when promising possibilities are identified, they should be exploited at a rate and intensity related to their estimated promise, which is continually updated. But at all times exploration for new possibilities should continue.
Therefore the principle is important for adaptive information processing in changing environments. It is also important in many other fields, for example in reinforcement learning and economics: should companies invest in research & development and fund innovative ideas or should they try to use their available resources and techniques more efficiently ? The principle of balancing exploitation and exploration is based on two core principles of evolution: exploration means variation (for example by recombination or mutation), while natural selection is similar to exploitation.
Swarm Intelligence
Systems which use swarm intelligence are often able to achieve the right balance between exploration and exploitation. During food foraging, colonies and swarms must achieve a good trade-off between exploration (search new sources) and exploitation (use known sources). Trail-based foraging amont ants and danced-based foraging among honeybees allow naturally to regulate the number of foragers: if the trail is weak and the food quality low, many ants will lose the connection, and start to explore new areas. The weaker the trail, the larger is the number of lost ants who are able to discover new food sources. If the number of dancers in a honeybee colony is low, the unemployed foragers will leave the colony and start to explore new areas, too. The lower the number of dances, the larger is the number of scouts which explore new areas.
Books
- Holland, J. H. 1992. Adaptation in Natural and Artificial Systems. Cambridge, MA: MIT Press. First edition, 1975
