The minimum description length principle is a formalization of Occam's Razor in which the best hypothesis for a given set of data is the one that leads to the largest compression of the data. In short, data that shows a lot of regularities and/or patterns, may be compressed without losing any important information. Applying this to learning, we can conclude that the more regularity and/or patterns we are able to find within data, the more we have learned about the data.

To illustrate this imagine the following as representations of two sets of data:

Set 1: 100110111011011001010110110010001100101101 Set 2: 011011011011011011011011011011011011011

Set 1 appears as to be random, but we with set 2 we are able to detect a pattern, thereby allowing us to describe it as "011 repeating 13 times.

For more information see: http://learningtheory.org/articles/mdlintro.pdf

[

See also

• Categorization
• Induction
• Rule induction

[

Footnotes

• Note 1: http://portal.acm.org/citation.cfm?id=66445
• Note 2: Dejong, Gerald and Raymond Mooney. "Explanation-based Learning: An Alternative View." Machine Learning. Volume 1. 1986. pp. 145-176.
• Note 3: See 1.
• Note 4: Ashcraft, Mark A. Cognition.Fourth Edition. Pearson Eduction, Inc: Upper Saddle River, NJ. 2006.
• Note 5: See 2.
• Note 6: http://dictionary.reference.com/browse/analogy

[

References

1. ^ http://en.wikipedia.org/wiki/ACT-R
2. ^ MIT : Brain and Cognitive Sciences : People : Faculty : Joshua Tenenbaum

• Ratcliff, Roger (2006). "Comparing Exemplar and Rule-Based Theories of Categorization". Current Directions in Psychological Science 15: 9–13. doi:10.1111/j.0963-7214.2006.00397.x. 
• A Rational Analysis of Rule-based Concept Learning. Retrieved on 2007-12-04.
• GA-based Rule Enhancement in Concept Learning. Retrieved on 2007-12-04.
• Bruner, J., Goodnow, J. J., & Austin, G. A. (1967). A study of thinking. New York: Science Editions.
• Feldman, Jacob (2003). "The Simplicity Principle in Human Concept Learning". Psychology Science 12: 227–232. 
• Rendell, Larry (1986). "A general framework for induction and a study of selective induction". Machine Learning 1: 177–226. doi:10.1007/BF00114117. 
• Watanabe, Satosi (1969). Knowing and Guessing: A Quantitative Study of Inference and Information. New York: Wiley. 
• Watanabe, Satosi (1985). Pattern Recognition: Human and Mechanical. New York: Wiley. 
• Solomonoff, R. J. (1964). "A formal theory of inductive inference. Part I". Information and Control 7 (1): 1–22. doi:10.1016/S0019-9958(64)90223-2. 
• Solomonoff, R. J. (1964). "A formal theory of inductive inference. Part II". Information and Control 7 (2): 224–254. doi:10.1016/S0019-9958(64)90131-7. 
• ACT-R. Retrieved on 2007-12-04.
• Brain and Cognitive Sciences. Massachusetts Institute of Technology. Retrieved on 2007-11-23.
• Kearsley, Greg (Copyright 1994-2007). Component Display Theory (M.D. Merrill). Retrieved on 2007-12-04.
• Kearsley, Greg (Copyright 1994-2007). Concept. Retrieved on 2007-12-04.
• Component Display Theory (2007-04-10). Retrieved on 2007-12-04.
• Concept Attainment (Copyright 1999). Retrieved on 2007-12-04.
• Concept Learning (2007-11-7). Retrieved on 2007-12-04.
• Concept Formation. The McGraw-Hill Companies (Copyright 2007). Retrieved on 2007-12-04.
• ⁶Berry, Donald A. (1997-1998). "Teaching Elementary Bayesian Statistics with Real Applications in Science". The American Statistician 5 (3): 241–246. 
• ⁷Brown, Harold I. (1994). "Reason, Judgment and Bayes's Law". Philosophy of Science 61 (3): 351–369. doi:10.1086/289808. 
• ⁵Lindley, Dennis V. (1983). "Theory and Practice of Bayesian Statistics". The Statistician 32 (1/2): 1–11. doi:10.2307/2987587. 

BCUZ.com FACTS Encyclopedia content is licensed under the GFDL as approved by Wikipedia.
For more information review our copyright contact and privacy policy. © 1996 - BCUZ.COM - We have all the FACTS you need about Small Business Financing, Behavior Disorder, Having Too Many Bills, Needing Cash Fast, Structured Settlements, Frequent Flier Programs, Top Steak Houses, The Mayan Indians, Norfolk and Suffolk England, Growing Longer Hair and a full reference English Encyclopedia and Spanish Encyclopedia.Privacy Policy