Information for MPW

As has been shown numerous times in the past, common sense is not a reliable judge of truth or falsity with regard to the fine-grained details of reality. Yours is no exception, and I shall demonstrate:Please read: http://home.mira.net/~reynella/debate/shannon.htmI expect that most of it will be over your head, but it's at least worth a shot for you to try to read it. If you garner nothing else from that page, at least find where it states that the formula for calculating information is:

Where I is information, and p_i is the probability of a selected element of the set of all the specified elements.As an example, say we have a set A of elements a₁, a₂, and a₃ input into a selection machine. This machine is designed to select one element from that set at random and out put it. We turn it on, and it pumps out a₂.Now, since we know that there were 3 elements out of which one was chosen at random, the probability of a₂ = 1/3.We can now calculate the information resulting from the selection of a₂ by using the formula:

Notice that the value is positive. Selection always results in information increase.Now, suppose that the elements a₁, a₂, and a₃ are alleles in the genome of an organism. If natural selection permits one of these alleles to emerge to the extinction of the others, then the calculation would be exactly the same, and information in the genome would have increased. You've conceded that selection operates on biological organisms, and it's been demonstrated in any case if you decide to reverse yourself.So now, please cease with your ludicrous claims that information does not increase through natural selection. ALL forms of selection produce information increases, as I have just shown.

Hi, Percy!Let me give this another shot. This all seems fine except for where you state that "each allele is a piece of information". Information is not a thing, per se. It does not have mass or dimension. It is a measure of the change in the probability of the appearance of a certain thing or symbol. It's like temperature -- something we abstract from the behavior of elements of external reality, but not an external reality itself. And this I disagree with.The greater the diversity of alleles, the greater the information gained if one were selected for. This is because as there are more and more alleles, the probability for the selection of any one allele gets smaller. Then, when one is selected for, we gain more information. In that sense, the more mutations there are, then the more information is increased as one allele establishes itself as the most advantageous. Gradually it will be selected for, however since it was selected out of a greater number of variants, it's probability is necessarily smaller and hence the information has increased by a greater factor.But what if a mutation occurs to an allele that also passes the selection filter? Say, a₅ is a mutation of a₂ yet also passes the selection filter. Then we have:-log₂(2/5) = 1.32 bitsIf that mutant allele had NOT been fit to survive the selection filter, we would've had:-log₂(1/5) = 2.32 bitsSo you can see that a greater number of mutations, if not deselected, can actually lower the degree of information increase. However, anywhere we have a finite number of alleles at one time, and then at a subsequent time we have a smaller number of alleles which survived based on fitness, selection has operated upon that set of alleles and the remaining ones carry an increase in information.It is hypothetically possible to have X number of alleles, pass them all along (i.e. do not select for any) and then add, say, 2 mutations to that set of alleles, and the result would be a information decrease. It seems that in reality, though, that selection acts much faster than mutation does (i.e. most mutations are disadvantageous and are generally deselected) and consequently information increases over time. Well, this is strictly true since once a single allele is selected and none remain, then the probability that it will be "selected" again is 1. In that case, I think calling it "selection" is a bit of a misnomer. If the number of unique alleles in a subsequent generation is greater than the number of alleles in the previous generation, then information would have decreased, but this is not "selection" per se.

Hi, Percy!It seems I'm still struggling to explain how I understand this to work. Please bear with me as I give it another shot. This all makes sense. However, the I don't see how this supports your labeling each allele a piece of information. Information is what is produced when a message is selected from the message set, but it is not the element of the set itself. I spoke of the selection of a single element for simplicity's sake. The principle still holds across a population. It's just that instead of 1 being selected from 5, we might have 345 selected from 347, or whatever. Agreed. Well, I did make the example up on the spot, but the principle is not ad hoc. Let me try to explain more thoroughly.If we had 4 elements in the message set {a₁, a₂, a₃, a₄} and the element selected was a₁, we will assume for the example that each element was equally probable and state the probability of a₁ as 1/4. Thus the formula states:

So the selection of element a₁ generated 2 bits of information.Now, say that these elements are alleles in a genome. Perhaps it will be easier to visualize if we increase the number of elements by a factor of 10. Say there are 4 different alleles for the same gene equally distributed across a population of 40 organisms. So we have 10 of a₁, 10 of a₂, 10 of a₃, and 10 of a₄.Now say all of the members of the first set pass along their traits to one descendent and then die, except one member has two offspring and a new allele appears (the offspring are an a₁ and an a₅). Meaning now we could have 1 of the a₅, 10 of a₁, 10 of a₂, 10 of a₃, and 10 of a₄. The probabilities shift so that the probability of a₅ is 1/41, the probability of a₁ is 10/41, a₂ is 10/41 and so on.Pass them through the filter, and suppose that again only the a₁'s pass through. Now we have:

So we can see that even the mere existence and deselection of the odd mutated allele increases the incremental information gain compared to before it was among the population in the previous generation.If the a₅ would have passed with the a₁'s instead of being deselected, the calculation would have been:

So what I was hoping to illustrate here is that a beneficial mutation (one that passes through the selection filter) can increase information once it is selected for (as in (3)), just not as much as if was deselected (as in (2)), and even less than if it had never existed in the message set (as in (1)).Does that make sense? Disregarding the meaning of the printed text on the pages, the selection of 1 page out of the entire set of pages would result in an increase in information. It seems intuitively wrong because you're conflating meaning with information. Shannon's paper states in the 2nd paragraph of the introduction:

quote:

---

Frequently the messages have meaning: that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem. The significant aspect is that the actual message is one selected from a set of possible messages. (emphasis in original)
From: http:///DataDropsite/Shannon.pdf

---

So I hope you can understand that it is valid, strictly speaking, to ignore that the pages happen to exhibit a meaningful arrangement of letters. I'm only speaking about the pages and their numbers. I don't think you've done the math properly.If there were 64 in the message set, and 32 were selected, then the probability that those 32 were selected is (32/64). The equation would thus state:

Or a 1 bit increase in information upon the selection of 32 elements of the message set out of 64.I said:

quote:

---

It is hypothetically possible to have X number of alleles, pass them all along (i.e. do not select for any) and then add, say, 2 mutations to that set of alleles, and the result would be a information decrease.

---

I worded that statement poorly. I meant to describe a relative information decrease. In other words, a greater number of alleles can result in a smaller gain of information than had the extra allele not existed, but not that the gain would be negative.So, how'd we do? [This message has been edited by ::, 02-03-2004][This message has been edited by ::, 02-03-2004]

BUMP!Hi, Percy.I'm bumping this up to remind myself to return to it and respond to your post. It's just that I've been short on the kind of time a response like this will require.

Hi, Percy. My apologies for the wait until I could supply my response. I think I've identified where we disagree with regard to the application of Shannon's Theory, and I'll try to clear this up before going back to explain my previous arguments. I don't think that this is valid and I'll explain why.In Shannon Theory, "information" is defined as a measure of a change in certainty or uncertainty. When you become more certain of the message, you've acquired information. When you become less certain of the message, you've lost information. Therefore, it is invalid to calculate the information "contained" in the entire message set prior to the selection and transmission of an element because there's been no change in certainty. Information isn't "contained" in the message set as it sits before a message is sent. It is abstracted from the process of selecting and sending a message.It could be argued, I suppose, that simply defining the message set reduces uncertainty and therefore supplies information, but if we were to define a message set of 5 elements out of the literally infinite number of possible elements that exist in the universe's message set prior to definition, our calcluation would be:

It is only when a message is selected out of a defined message set that we can calculate a meaningful probability of that message and therefore measure the reduction of uncertainty. The receiver does not need to know which message will be sent in order to get information; he must only know which messages are possible, or what messages are elements of the defined message set.Before a message is sent, the receiver has complete uncertainty. He does not know which of the possible message will be sent. If he receives 2 of the 5 possible messages, his uncertainty is reduced and he has acquired information. However, if he were to receive only 1 of the 5 possible messages, his uncertainty would be reduced by a greater degree, and that is why the transmission of 1 message out of 5 transmits a greater measure of information than a transmission of 2 messages out of 5.This is where the confusion lies, IMHO. Do you see what I'm getting at?

Let's look at the general formula:

...where p_i is the probability of the transmitted message(s). To calculate p_i, you simply express it as the number of elements of the message set that were transmitted divided by the total number of possible messages in the set. If we had a message set of five possible messages, and two messages were included in a transmission, the probability that those two messages would be sent is 2/5, or p_i = 2/5. Then we can calculate the information gained in the transmission with the formula like so:

I agree that the message set is always predetermined, and my example of trying to calculate the selection of 5 messages out of infinite possible messages I think distracted from the real point which is that there is no meaningful calculation of information over an entire message set as a whole. Basically what I was trying to illustrate is that in defining a message set, you're selecting 5 messages out of an infinity of possible messages (because they could be literally anything), and that since the definition of the set is somewhat like a selection process, one might think that a meaningful calculation of information could be made over the set as a whole. So in my calculation, 5/∞ was supposed to be the value of p_i, the probability that 5 messages would be selected out of an infinity of possible messages. Actually, more than one message can be sent in a single transmission, however uncertainty is not reduced as much as if only one message were sent. That's why you gain less information taking the negative log base 2 of 2/5 than you get taking the negative log base 2 of 1/5.Does that help?[This message has been edited by ::, 02-12-2004]

Alright, cool... it looks like we're making some good progress toward understanding eachother. I'm putting this response here to let you know that I've seen your most rencent posts, although I haven't really gone over them in depth. I don't think I'll be able to really get back into this until early next week, but I do intend to continue.Have a great weekend and take care,
::