Information Theory and Music
If “Your Brain on Music” is any indication, one resource the field appears to be under-using to date is Claude Shannon’s information theory. Shannon defines an information theoretic “bit” to be the amount of information needed to answer one YES/NO question about which you are initially entirely undecided.
(Information-theoretic “bits” are NOT to be confused with vanilla storage “bits” — the coincidence in naming is hugely unfortunate but apparently incurable. A blank blu-ray disk holds about fifty billion “bits” of storage but zero “bits” of information, because you know ahead of time that every single one of them is definitely zero — actually reading the disk yields you NO new information.) If a tune starts out with a phrase and repeats it endlessly, exactly, in information theoretic terms what you see is a burst of information reception at the start as the phrase is learned, followed by an exponential drop-off in the information-theoretic bitrate as it becomes more and more probable to the listener that the phrase is repeating exactly, and that each note is now almost entirely predictable, and thus carries near-zero information.
Maintaining a constant INFORMATION bitrate thus requires constant variation of some sort (almost any sort) in the phrase to re-introduce uncertainty — which to my mind sounds just about exactly like what one in fact observes in music of all kinds.
Qualitative music-theory talk about “building excitement” and such circle around this idea, but do not allow quantitative theorization and experimentation: They are like the sterile old qualitative theories of gravity which held that objects fall faster as they approach the Earth out of gladness to be returning home — real progress in the field only began with Galileo’s quantitative work.
Information-theoretic bitrate (formally, “Kolmogorov complexity”) is in principle intractable to compute, but in practice in most cases is surprisingly well approximated by state of the art compression programs such as the research program PAQ, or even by vanilla run-of-the-mill compression programs like ppmd (Russian markov-chain compression), bzip2 (block compression) or in a pinch even gzip (classical Lempel-Ziv style compression).
One could, for example, try using such a program to compress a variety of MIDI files (taking MIDI as a trial approximation of the skeletal psychological structure of music after human preprocessing has done tone resolution &tc) and look at the average information-theoretic bitrates: I would not be surprised
if it turned out that they cluster closely, possibly around the 2 bits/sec number that Minsky cited as the limit on write rate into human long-term memory.
As a follow-up, by appropriately instrumenting a markov-chain based compression program like ppmd, it would be entirely practical to plot information content on a note-by-note basis, allowing study of how much and how information content varies through a piece. It turns out that such variations can be used to recognize phrase boundaries from first principles in text (uncertainty pops up to a maximal value at phrase boundaries and drops to a minimum near the end of a phrase) and the same is almost certainly also true of musical phrases. (Some nice Israeli work on grammar induction via multiscale markov pattern recognition came out about four years ago, although I cannot find the reference at the moment. 
For an excellent example of using compression programs as the general-purpose pattern recognition systems which they in fact necessarily are (a widely under-appreciated fact — the only way to compress a file is to recognise and exploit patterns in it) is Rudi Cilibrasi’s thesis (?) work under Paul Vitanyi: See for example Clustering by Compression (pdf file) in which various types of music are plausibly ranked by similarity from first principles using differential co-compressability (with similar clustering done the same way on a variety of other datasets ranging from DNA sequences to text).
Filed under Uncategorized | Comment (0)Getting it together
This is it: Sandy and Cynbe’s blog. It’ll take a few days to get organized, but at least we’ve got the basics goin’.
Filed under Uncategorized, About us | Comment (0)