Evolving New Information

Having a life is consistent with the delay, but not with the constant attempts to avoid explaining how you calculated your information values. Put them together and it adds up to evasion. How did you calculate the figure of 852 bits?--Percy

Hi LucyTheApe,Using this image that you provided: Please describe for us how you calculate an information content of 832 bits.--Percy

As I explained back in Message 367, the size of the file output by a compiler is not a measure of a program's information. The size of that output file will vary by compiler version and platform, but the amount of information in your program must be a constant, so obviously compiler output is not a measure of information.You're obviously very confused. First you claimed that you used a compiler to calculate the information, then in response to my explanation that this is not a method by which one can measure information you denied using a compiler and accused me making that erroneous claim, then you became evasive for several months, and now you're claiming you used a compiler again.You can't use a compiler to calculate the amount of information in a computer program. An accurate calculation would be a very complicated exercise. A compiler is really just a translator from one language (the computer programming language) to another language (the binary instructions of a computer's CPU).You need to either present an example simple enough that you can calculate the amount of information, or accept the example I presented earlier in this thread, which has the additional advantage of using real DNA nucleotides and codons, see Message 1.--Percy

I don't think I ever said reality is information. I would be more likely to say something like, "Everything in reality is exchanging information with everything else in reality all the time."But I didn't say the output file of a compiler does not contain information. What I said, in a bit more detail this time, is that the number of bits in that output file is not a measure of its information content.By the way, all compilers that I'm aware of generate output files in units of bytes (more commonly, units of 4 bytes, often called words), so the number of bytes in any compiled file should always be a whole integer. Since 852 bits is not an integer number of bytes (it's 106.5 bytes), I very much doubt that your Java compiler produced a file 852 bits in length. Did you perhaps mean 852 bytes?In any event, your program must contain a fixed amount of information. Whatever that amount is, I hope we both agree that all valid methods of measuring its information content must yield the same value. When a carpenter measures a board he expects its length to measure the same whether he uses a ruler, a yard stick, a tape measure, or the rule on his T-Square. And analogously, when we calculate the information content of something, we expect that the value calculated will not vary, no matter what method we use, as long as it's a valid method.So when you say this: You've already conceded the point. You acknowledge that different compilers will yield different measures of the amount of information in your program, and therefore compilers by their inherent character cannot be valid measures of information content because they provide a variety of different values for the exact same program.By the same token, the amount of information in DNA is not a variable function of the measurement method. If the DNA sequence doesn't change, then the amount of information in it cannot change, no matter who measures it and no matter what method they use. This is another reason why you should question your compiler example. If compiler output is a valid analog to DNA for the sake of this discussion then its information content doesn't change if you use a different compiler. Or if a program's information content *does* change according to compiler (which would make no sense, I'm just including the flip side of the argument), then a computer program is not a valid analog with DNA.In case you found that last point confusing, just remember that a valid information measurement method will always give the same answer for the same input.--Percy

Hi LucyTheApe,Well, guess you're off living your life again, so since I don't know whether you're going to reply in a day or a month I'll post some thoughts I've had about where you might be going wrong while I still remember them.I think you must believe that the number of bits used to save data in a file is the amount of information in that data. This is wrong. Let me explain why using your Java shell as an example: However much information exists in this program, it must be a constant. We don't know how much information it has, but unless you change it, which we're not going to do, it must be constant. There is only one value for the amount of information in your program shell. All valid methods of measuring its information content must yield the same value.So let's say we use Java compiler X to compile your program, and it produces a file 1200 bytes in length.And then we use Java compiler Y to compile your program, and it produces a file 1300 bytes in length.If compiled file size is the amount of information in your program, then how much information does your program contain? Is it 1200 bytes or 1300 bytes? Only one answer can be correct, right? And is it possible neither answer is right? You bet!File size is also not a measure of information content. For example, you could save the first paragraph of this message in a file and see that the file size is 218 bytes including the newline character. But you could then run the file through gzip and find that it is now only 177 bytes in length. Gzip relies upon redundancy to compress files. So how much information is in the file? Is it 218 bytes or 177 bytes? Only one answer can be correct, right? And is it again possible neither answer is right? You bet!Once you accept that compilers and file size are not a measure of information content, then we can continue on to talk about how one goes about measuring information content in a valid way.--Percy

You keep saying this. I keep explaining why the size of the file output by a compiler is not a measure of the amount of information in the original program, and you keep replying with restatements of your assertion without explanation. Boy, talk about missing the point! The length of a board was an example of measurement, not an analogy with information. When we measure anything, whether the length of a board, the amount of water in a cup, the weight of a sample, we expect those measurements to always be same, as long as we're using valid measurement tools.That the size of your compiled file varies from one version of the compiler to the next and from one platform to the next is telling you, unequivocally, that a compiler does not provide a measure of the amount of information in your program. Your measurement of a board will not change when you change rulers because a ruler is a valid measurement tool for length. If compilers were a valid measurement tool for information then the amount of information in your program cannot change just because you change compilers. But it does change when you change compilers, and so compilers cannot be a valid measurement tool for information.There's even the questions of why you think the compiler output is the amount of information in your program in the first place? Why is not the original program file the amount of information? You would still be wrong, but this approach at least has the saving grace of always giving the same answer.The size of a file is not a measure of the amount of information in that file. To measure the amount of information you have to unambiguously define your message set. If you put the DNA sequence CAGTTC in a file and found that it was 6 bytes long, would you then claim that that sequence contained 6 bytes of information? No, of course not. So why are you claiming that when you place compiler output in a file that the file size is the amount of information in the original program.--Percy

First, let's be very clear about one thing: you don't know how to calculate the information content of a dataset. Independent of whether your definition of information, "a coded message," is correct, it doesn't provide you any method of quantifying information. This was effectively made clear by your recourse to compilers as information measurement tools, despite that they give variable answers for the same program file.I use Shannon information to calculate the amount of information in a set of data. This requires defining the message set, because communication of information requires reproducing at point B a message that originated at point A.If our message set is 0 and 1, a message set of size 2, then the amount of information that can be communicated in a single message is log₂2=1 bitIf our message set is the digits from 0 to 9, a message set of size 10, then the amount of information that can be communicated in a single message is log₂10=3.32 bits.If our message set is the letters from A to Z plus space, a message set of size 27, then the amount of information that can be communicated in a single message is log₂27=4.75 bits.The reason we don't use programming examples to explain information theory is because a programming language is so incredibly complex because the message set is context dependent. For example, ask yourself what is the message set for the first field of your program. Here's a partial list:

• //
• /*
• public
• private
• import
• ...

I don't know how many messages are possible for that first field, but let's assign it a value X. So the information communicated by the first field of a Java program is log₂X bits.Now ask yourself what is the message set for the second field of your program. Well, that depends upon what the first field was. If the first field began a comment then pretty much anything goes until the end of the comment field. If the first field was "public" or "private" then the message "class" might follow, and there are other possibilities. So the information communicated by the second field is a function of the first field, we can call it log₂f(field1) bits. That's what I meant by context dependent.The third field's message set will be a function of the first two fields, and the fourth field's message set will be a function of the first three fields, and so forth. You can see that this gets complicated very quickly.This is why you were immediately challenged when you claimed you knew how much information was contained in your Java program. You had already demonstrated you knew very little about information theory, yet here you were claiming you could solve the very, very complex problem of calculating how much information was in a simple program. We knew it couldn't be true.This is why I suggest we talk about examples where we can actually calculate the information content. The example from Message 1 fulfills this requirement. Here's a message set of size 3 that for the sake of discussion we can say represents the alleles for three different eye colors in a population:

• GGAACG (green eyes)
• GGAACA (blue eyes)
• GGCACG (yellow eyes)

If one individual in our population receives a mutation, then our population now has four alleles for eye color:

• GGAACG (green eyes)
• GGAACA (blue eyes)
• GGCACG (yellow eyes)
• GGCACA (brown eyes)

The amount of information in our population for eye color has just increased from log₂3 = 1.585 bits to log₂4 = 2 bits, an increase of .415 bits. Mutations are one way of increasing the amount of information in the genome of a population.--Percy