Information Theory is an umbrella term for the scientific disciplines that attempt to codify the mathematical underpinnings of data. In particular, Information Theory is interested in topics such as data compression, data communications, and error correction
Links to the articles from this journal dating back to 1953!
(As former Trans. of the IRE professional group on comm. systems.)
Quite a number of compression-related articles were published
in this title, although it aims at channel coding problems. The bad news is that you are only allowed to see the abstracts if you're not an IEEE member.
Created: 10/10/2005
by Maxim SmirnovMore...
How much can a given piece of data be compressed? This FAQ addresses that question, including some necessary critiques of the question itself. Graham Fyffe apparently spends some time thinking about complexity.
Created: 29/07/2000
by Mark NelsonMore...
An introduction to Shannon's notion of probabilistic entropy, from the viewpoint of ergodic theory and dynamical systems theory. Includes a complete proof a reasonably general (and useful!) version of the Asymptotic Equipartition Principle (AEP). Includes figures.
Created: 12/10/2002
by Mark NelsonMore...
Links to software, organized into three categories: Tutorials, Freeware, and Payware. Organized by Chris Hillman.
Created: 12/10/2002
by Mark NelsonMore...
An enormous page of data compression links, well organized by type. Although this page is hosted in Japan, nearly all of the links are described in English.
Created: 06/11/1999
by Mark NelsonMore...
Quite a number of information theory classic papers. The parent node http://cg.ensmp.fr/~vert/proj/bibli/ has additional related articles.
Created: 13/06/2005
by Maxim SmirnovMore...
An article by Eric Uner talks a bit about generating reandom numbers, something we all want to do from time to time.
Created: 20/06/2004
by Mark NelsonMore...
This page describes a program, ent, which applies various tests to sequences of bytes stored in files and reports the results of those tests. The program is useful for those evaluating pseudorandom number generators for encryption and statistical sampling applications, compression algorithms, and other applications where the information density of a file is of interest
Created: 13/06/2004
by Mark NelsonMore...
This web site offers you random numbers generated using quantum process, which sounds like the gold standard to me.
Created: 04/04/2004
by Mark NelsonMore...
This book by David MacKay can be downloaded chapter by chapter and printed out for free. No indication that you can buy a hardcopy, although Amazon.com claims it will be published in January, 2002. Please read this 600 page tome and provide a review for this space!
Update: The author tells me this book will be published in November, 2003. The book will remain viewable on his website!
Created: 04/06/2003
by Mark NelsonMore...
Current research projects in our lab include research into vector quantization (VQ), wavelets, image compression, edge detection using VQ, VQ for image browsing, VQ design for noisy channels, halftoning, and color palette management.
Created: 04/05/2003
by Mark NelsonMore...
The University of Washington has a nice bibliography here, with pointers to books on Data Compression, VQ, Wavelets, and Information Theory.
Created: 04/05/2003
by Mark NelsonMore...
Tjalling is on the faculty of Technische Universiteit Eindhoven, which appears to be located in Eindhoven, The Netherlands. He is involved in research of various aspects of Information Theory, including Context Tree Weighting and Coding.
Created: 01/04/2003
by Mark NelsonMore...
The whole contents of the journal, dating back to 1953. If only you were a member of the society, you could read all these great papers.
Created: 21/03/2003
by Mark NelsonMore...
Links to the articles from this journal dating back to 1988. If you're not an IEEE member, you only get to see the abstracts! Membership in the IEEE plus a few extra bucks lets you download here to your heart's content.
Update: Site has moved due to redesign.
Created: 03/09/2002
by Mark NelsonMore...
by Peter Fenwick, Department of Computer Science, The University of Auckland.
In his work on the information content of English text in 1951, Shannon described a method of recoding the input text, a technique which has apparently lain dormant for the ensuing 45 years. Whereas traditional compressors exploit symbol frequencies and symbol contexts, Shannon's method adds the concept of "symbol ranking", as in `the next symbol is the one third most likely in the present context'.
Created: 19/01/2002
by Mark NelsonMore...
An article in the New York Times that gives a thumbnail description of Claude Shannon. You will need to register with the Times in order to read the article.
Created: 30/12/2001
by Mark NelsonMore...
A set of handouts for a survey course on Data Compression, inlcuding papers on various types of coding, information theory, and more. An academic approach to the topics but quite accessible.
Created: 05/11/2001
by Mark NelsonMore...
The home page for the society. Links to newsletters, conferences, tutorials, abstracts, and other miscellaneous fun things.
Created: 27/09/2001
by Mark NelsonMore...
This issue of the newsletter talks about randomness, complexity, and a long-time favorite, recursive compression.
Created: 11/08/2001
by Mark NelsonMore...
by Robert B. Ash. This book gets a couple of five star ratings on Amazon.com. If you're the type to pay close attention to your budget, you'll like the fact that it sells for under $10!
Created: 11/08/2001
by Mark NelsonMore...
Robert X. Cringely waxes on a bit about the death of Claude Shannon. Bob definitely gives the Father his props.
Created: 04/03/2001
by Mark NelsonMore...
This bibliography is the best attempt at a complete record of all the written works of Claude Shannon, father of Information Theory.
Created: 23/09/2000
by Mark NelsonMore...
A bronze statue of Claude Shannon will be unveiled on Friday, October 6, 2000 in his hometown, Gaylord, Michigan. This sculpture by Eugene Daub was commissioned by the Information Theory Society of the Institute of Electrical and Electronics Engineers (IEEE). It will be installed in the recently named Shannon Park in downtown Gaylord.
Created: 15/09/2000
by Mark NelsonMore...
A very gentle introduction to Shannon's discrete entropy, written for moleculary biologists.
Reader Mike said To the point explanation of the "why" of some of the math. This allows you to look a symbols and understand the _meaning_ not just the math..
Created: 27/08/2000
by Mark NelsonMore...
by Claude Shannon. This paper is generally credited with being the opening kickoff of the science of Information Theory.
Created: 29/07/2000
by Mark NelsonMore...
According to the publisher, this contains all of Shannon's published works, plus papers that were previously unpublished for various reasons, including wartime secrecy.
Created: 29/07/2000
by Mark NelsonMore...
by Thomas M. Cover, Joy A. Thomas. This is an introductory text on Info Theory, covering all the basics, which of course includes coding and data compression.
Created: 29/07/2000
by Mark NelsonMore...
The comp.compression newsgroup regularly sees posts from folks with incredible compression claims. This one, started by Itai Bar-Haim, is remarkable not so much for its claims, but rather for the incredible length of the thread it spawned. All Atai said was that mathematical combinations have nothing to do with compression. So there.
Created: 27/06/2000
by Mark NelsonMore...
Home page for the book: Introduction to
Information Theory and Data Compression
Darrel Hankerson, Greg A. Harris, and Peter D. Johnson Jr.
Created: 03/06/2000
by Mark NelsonMore...
Charles Bloom puts the noggin to work on this question. It's not as foolish as the title might make it sound.
Created: 06/01/2000
by Mark NelsonMore...
by Hankersson et. al. Hardcover - 330 pages (November 1997) CRC Press; ISBN: 0849339855. A textbook for upper-level or graduate course on compression. Please use the link on this page to purchase this book, it will help support the Data Compression Library.
Created: 24/11/1999
by Mark NelsonMore...
The purpose of this applet is to help develop intuition about maximum entropy distributions. A central advantage of the maximum entropy framework is its ability to model overlapping events without increasing the number of parameters or fragmenting the training data. Using maximum entropy techniques, it is possible to model a probability distribution on 2n elements using only n free parameters.
Created: 22/11/1999
by Mark NelsonMore...
A journal that publishes papers dealing with randomness, pseudorandomness, and Kolmogorov complexity, among other things. Their home page features an on line listing of every paper published in this journal since 1957, with a search tool.
Created: 14/11/1999
by Mark NelsonMore...
8 lectures by David J.C. MacKay
January 1995. Cavendish Laboratory, Cambridge, Great Britain.
All of these lectures are available as PostScript files. The eight lectures cover data compression, coding, arithmetic coding, noisy channel coding, and much more
Created: 07/11/1999
by Mark NelsonMore...
Home page for John C. Kieffer, Dept. of EE, University of Minnesota. Includes course descriptions for classes on data compression and information theory. The course descriptions have links to some papers.
Created: 07/11/1999
by Mark NelsonMore...
Gerry Wolf's home page, including links to "Language Learning as Compression", "Computing as Compression", and "Cognition as Compression"
Created: 06/11/1999
by Mark NelsonMore...
Lecture notes for a course on Information Theory, by John Lafferty, School of Computer Science, Carnegie Mellon University. Includes handouts, problem sets, code, additional lectures, and a page of links.
Created: 24/12/1998
by Mark NelsonMore...
A reprint of an important paper. This site has links to the paper in PDF and Postscript formats. Claude E. Shannon is widely acknowledged to be the father of Information Theory. The publication of this paper established that reputation and gave birth to this area of scientific endeavor.
Created: 04/12/1998
by Mark NelsonMore...
A page of links for the authors of "Introduction to Information Theory and Data Compression," and contains pointers to materials related to data compression.
Created: 01/01/1970
by Mark NelsonMore...
