Information for MPW

Hi, :ae:!I wasn't able to follow your argument, even after looking at your link. It wasn't clear to me how natural selection can increase information.I approach the problem from a slightly different angle. The total information for any gene in a population of organisms is equal to, keeping it simple, the log base 2 of the number of alleles (the "keeping it simple" part means I'm assuming equal probability for all alleles, and that each allele is a piece of information). The total information in the population for this gene can only change if the number of alleles increases or decreases. The selection by any individual reproductive act of one particular allele neither increases nor decreases the number of alleles in the population. An increase can only happen through mutation, and a decrease can only happen when no offspring in a generation inherits the allele (call this allele death- there's probably a correct term for it, but I don't know what it is).Natural selection without mutation or allele death can neither increase nor decrease information. Even when you consider multiple genes working in concert permutationally, information cannot be considered to have increased or decreased during the gene mixing of reproduction because the log base 2 of all possible permutations is the total information in the genome, and individual expressions of these permutations do not affect it.At least that's the way I see it, but let me know what you think.--Percy

Hi, :ae:!I'm afraid I'm still not reading you. I was trying to use informal wording. In Shannon information terms (see Shannon's Original Paper), the total alleles for a gene in a population represent the total set of messages that can be copied to offspring. Each allele represents a single member of the total message set of all possible alleles for that gene in the population. You're thinking of the individual offspring as a receiver of information received from the parents, but it is only relevant to talk about the information in a population, and not of the information flow during the reproductive act - that's just copying. If it were true that the mere act of reproduction increased the information in a population, then a population could increase genomic information simple by increasing the size of the population. But having multiple copies of the same set of alleles doesn't increase the amount of information a population possesses, no more than possessing two copies of Shannon's paper increases the amount of information available to you. I don't know if this is just too profound for me, or if you're just making it up as you go along, but this makes no sense to me. You're going to have to explain this one. Clearly wrong, even just intuitively - just try creating information by ripping pages out of a book and burning them, and then by your logic when the book is empty it contains more information than it ever did.You can demonstrate this for yourself with a simpel example. If your population begins with 64 alleles and then later has only 32 alleles, then just do the math. The information present at the beginning for this gene was 6, and later it was 5. Again, clearly wrong. Keep in mind that when you add alleles to a gene you're increasing the message set size M, and that information, which is log₂M (or -log₂(1/M), whichever you prefer), increases with increasing M.--Percy

Hi, Tagless,I don't think I can be any more clear than what I originally said: I don't know why you ask "Are you referring to chromosome counts as information?", because I'm clearly talking about alleles for a single gene.--Percy

1 gene = 1 protein = 1 function = something I didn't say--Percy

As I already said, I was trying to use non-technical terms. Calling an allele a piece of information was not intended to convey anything more than that the allele is one message of a set of possible messages. While you might not like my word choices when expressing this informally, I was trying to consider the wider audience, and I *do* think they brought the right images to mind when read by those unfamiliar with information theory.Keep in mind that the Creationist argument is that evolution (meaning in this case reproduction and mutation) cannot increase information. This is clearly wrong. If the allele set size for a gene in a population is 8, then I=3. If there's a mutation and the allele set size grows to 9, then I=3.12. Creationist argument falsified.But you're making a different argument. You've even moved the argument from one of the amount of genomic information in a population to one of the probability of particular alleles being inherited. While this is certainly extremely relevant to population genetics, it is not the same issue of information usually raised by Creationists.I think I said a couple of times that I was keeping things simple (in keeping with the wider audience again) and assuming equal probabilities for all alleles, but your example uses unequal probabilities. I'll speak to this, but we may lose the Creationist audience: You're measuring the information (number of bits) *after* it's arrived. You must instead look at the total number of messages in the set *before* the message is sent, which is -log₂(1/5) = 2.32 bits. This is the minimum number of bits necessary to communicate messages of set size 5. Your answer of 2.04 bits is incorrect because the receiver of the information did not know which allele he would receive, and so there has to be provision that he could receive any of the 5 alleles in the message set. An information gain or loss occurs when the allele set grows or shrinks in size. You're incorrectly equating the specific information communicated to offspring with information measures. No, I'm definitely not "conflating meaning with information" - keep in mind I'm assuming Creationists are trying to follow this thread. That I long ago understood the difference between meaning and information can be seen in Message 74 of the old thread Information and Genetics where I discuss this with Dillan. I think there are any number of ways to consider a book as containing information (not meaning) - let me know if examples would help. All you've actually done is calculated the difference between the number of bits needed to communicate messages of set size 64 versus 32: That one bit is in no way a measure of any information actually communicated, or of the information in the population for that gene. The idea that you can just magically place the number selected in your numerator does not appear to have a justification or correspond to any real world situation that I can see.--Percy

Hi, :ae:!I'm afraid what you're saying still makes no sense to me. I'll just focus on a small part: When you say, "Simply defining the message set reduces the uncertainty," I think it reflects a fundamental misunderstanding of information theory. The message set is *always* predefined.What the sender wants to communicate to the receiver is not the message set, because they have pre-agreed on that, but individual messages from the message set. It isn't that the receiver doesn't know the message set, because he most certainly does! What he doesn't know, what is uncertain, is which message of the message set will be sent next.You also need to explain why you keep taking the log base 2 of any number that strikes your fancy, in this case 5 over infinity.--Percy [Minor phrasing improvement. --Percy][This message has been edited by Admin, 02-12-2004]

It is possible that you're trying to make one point while I'm making another, but most aspects of your position seem wrong to me:

1. Information does not increase with decreasing message set size.
   ​
2. Message sets are not transmitted. Messages are transmitted, one at a time, from the message set.
   ​
3. Your fractions of 2/5 and and so forth have no real world meaning because they do not correspond to any actual channel of information.
   ​
4. The size of an information channel is determined a priori by your message set size (and other characteristics), and not post facto by the number of messages received.

Selecting an allele for a gene during reproduction is equivalent to sending one message. The number of alleles in the population is the size of the message set for that gene. As Shannon says in his paper right on page 1, "The significant aspect is that the actual message is one selected from a set of possible messages." It is significant that Shannon goes on to deny your post facto claim in the very next sentence when he says, "The system must be designed to operate for each possible selection, not just the one which will actually be chosen since this is unknown at the time of design." Common sense tells us the same thing.In other words, the communication channel is designed to send one message from the message set. The communications channel must be of a size capable of sending one message at a time, and you can send consecutive messages in order to build more complex higher-level messages.An example: the ASCII character code. It contains 256 different characters, and requires log₂256=8 bits of channel. A message consists of a single character. You can send consecutive messages (characters) in order to send higher level messages of greater complexity.The alleles for a gene of a population can be thought of in the same way. The message set size is equal to the number of different alleles in the population, call it a. The minimum number of bits of communications channel necessary to communicate a single message of message set size a is log₂a. That means each organism needs at least log₂a bits of capacity to store a single allele, and we can think of the minimum bit capacity as a measure of the amount of information present in the entire population for that gene location. We can think of this gene location as a communication channel for transmitting (through the reproductive process) the allele to the next generation.The analogy with the ASCII character code breaks down after that point. With heredity, more complex messages are sent through the addition of more genes with their own set of alleles, and not by sending more alleles of the same gene. But the principle should still be clear.--Percy

Read what I said again. I'm talking about population allele set size: It doesn't matter how many individuals in the population have the "this" allele, it still only counts one toward allele set size.--Percy

I was thinking about this some more, and I think I see now where I'm having the biggest problem with your approach. This is from your Message 12 (note that I've changed terminology form Shannon's "message" to your link's "symbol", which seems better suited): What you describe in qualitative terms, both here and elsewhere, about the probability of symbols in our symbol set is correct, but your math is nonsense. Check out Shannon's original paper, or reread your own link, especially section Information, Entropy, and Uncertainty. You won't see anything like the games you're playing with your numerator. Determining the necessary channel width in bits for a symbol set of unequal probability is not a straightforward operation. It certainly isn't equal to the log base 2 of the probability of the particular symbol just sent.You may be doing this because you're confusing the information actually transmitted (eg, alleles inherited by an offspring) with communication channel capacity. You are correct that the minimum number of bits is much smaller for a very likely allele, but it is *not* the same thing as the minimum channel width necessary to transmit any allele of the set.Concerning unequal probabilities of symbols, this portion from your link is applicable: Obviously you know this already, but note that heredity doesn't really work this way, at least not at the simple level that we should initially approach it. Just because a particular allele overwhelmingly dominates the rest and is 99% likely has no impact on the code length necessary to represent that allele. The gene of an individual must be capable of representing the most unlikely allele in the population. The gene of an individual is a static record, not a symbol in a message stream. We cannot gain the efficiencies of special codings because that cannot be done across generations (ie, that the only way you could completely decode the allele of an organism is to know the coding for the alleles of the ancestors back one or more generations - coding is a science in itself - many is the time that Shannon's equations have said that some symbol set with some probabilities can be represented in some incredibly small number of bits, but the equations don't tell you the encoding, only that it's possible, and there's the catch). The gene location doesn't change its size in each individual organism based on the likelihood of the inherited allele (in reality I wouldn't put it past nature to contain examples of doing just this, but we want to stick with a simple model for now).So we can only conclude that the most reasonable measure of information in a population of organisms is the log base 2 of the number of alleles for each gene, independent of the probability of those alleles because our channel capacity is fixed, at least for the scenarios we are considering at this time.--Percy

That's your rebuttal - you doubt it? If you really think Shannon's definition is simple, or that it is insufficiently nuanced to represent genetic information, then I suggest you read his paper.Your approach in Message 21 of that other thread lacks rigour, has a confused presentation, and I'm not convinced time and energy invested in its decipherment would be repaid.--Percy

I said that each allele of a gene in a population of organisms is a symbol of the symbol set, and the symbol set size a is equal to the number of unique alleles for that gene in the population. The information measure for that gene is I=log₂a. That seems fairly specific.--Percy

Hi, :ae:!Saviourmachine's messages prompted another thought.Shannon information isn't really a measure of information, but only a measure of the minimum bits required to represent or transmit members of a symbol set. You tend toward examples where the set members have unequal probabilies, and this can yield very small values for I relative to symbol set size.But allele frequency in a population changes over time, while the genetic code for the alleles of a gene is fixed, so we must pessimistically assume equal probabilities for all set members, which always yields the largest values for I.We must also keep in mind that the Shannon equations yield a minimum possible value. Genetic codes are nowhere near so dense, though some of the redundancy probably contributes to error tolerance.--Percy