can personal bias be 100% eliminated with training?

In another thread holmes made what was, to me, a rather outrageous claim regarding his ability to evaluate resarch studies.He claimed that because he has training, he is completely immune to personal bias, as seen below:

quote:

---

That is true, that was my training, so what?

---

If this is true, then does that mean that all people who have similar training to holmes are equally immune to all personal bias when evaluating research?Exactly what was your training, holmes, that makes you immune to bias in this way?

Theoretically yes. In practice it will depend on your ability to stick with your training.But let me clear something up. I did not claim to be completely immune to personal bias of any kind. You originally made a very specific claim that I could decide a study's conclusions were errant because I did not like what they said. That is a very specific type of bias and one I am immune to. If your later statement had a broader meaning then you were making a strawman of my position.Here is how I do show bias (as I am not perfect) when reviewing studies:1) If a study's results sounds reasonable (consistent with other research I am aware of) I am less likely to look it over, than other studies which sound less reasonable.2) When I do not like an author or an organization or was told ahead of time that it reached a conclusion I do not like, I am more likely to to go over it with an exceedingly fine toothed comb to find every error possible. I will wish to portray it in as negative a light as I can legitimately find, even if it is not necessary.Thus I am prone to accepting erroneous material when I have reason to like it, because I am more prone to giving it a break. I am also prone to continue kicking a study that I hate when it is already down. Not that the kicks aren't legitimate, just that I deliver all the bad news, rather than just delivering the sufficiently bad news.In any case, whether I like it or not, when I am asked to review a study then I do it with a very critical eye. I have told you in specific in earlier threads where I used social studies to support my own position, that I had criticisms of them. Thus, even for stuff I like I can be a harsh reader. I'm not going to hand out my cv. In general I have had a few courses in research methodology (both hard and social sciences) as well as work on spotting logical errors within scientific papers.In addition to grad work where I had to analyze papers, I went into work where I had to spot errors in scientific "papers"... hard to call it research, more like pre research. I also had to deal with scientists on a near daily basis regarding data and sometimes that went into analysis of data, on a rather massive project.There's more but that's enough.The training was straightforward and the expectation was neutrality toward conclusion, to analyze methodology. I am not so sure why that would be hard to understand.If a study has valid methodology but conclusions I do not like, then it is time to address the next step of the "problem" as I might see it because the data is there.I guess the one thing I learned, even more important then methodology, is that the history of science proves good data wins out eventually. I mean there can be periods of history where people simply ignore the data, but eventually data wins out. It is silly to try and argue away data that will keep coming back. Better to accept it and move on.I had a teacher that was against the theory of continental drift. Of course it is humorous now, but his story was a great lesson in how to keep an appropriately open mind and not stick to one scientific theory as dogma.This message has been edited by holmes, 08-09-2005 03:44 PM

---

holmes
"...what a fool believes he sees, no wise man has the power to reason away.."(D. Bros)

I'm at work so I can't make a full reply, and I can't reply in the other thread anyway, but I have an answer to this:

quote:

---

She creates one thread insinuating science cannot possibly be handled without bias,

---

No, I did not say that.I implied that it was impossible that individuals were not completely, or even mostly, free of presonal bias.Everybody has a different awareness of personal bias, and differing amount of bias, and bias regarding different aspects of different subjects and issues.Anyone who has published scientific papers knows that the comments from the reviewers are almost never identical, and sometimes they can be clearly biased.This is why there are lots of journals, with lots of different editors, and the editors change all the time, and things are done as anonymously as possible, etc.Scientific progress is made through consensus, and that is a major way personal bias is dealt with.

Excuse me for asking but what's IIRC and what is cv?

IIRC -- if I recall correctly
cv -- curriculum vitae (loosely Story of your life ) or resume

Thanks Ned.

quote:

---

You originally made a very specific claim that I could decide a study's conclusions were errant because I did not like what they said. That is a very specific type of bias and one I am immune to.

---

How do you know you are immune to this kind of bias?

quote:

---

The training was straightforward and the expectation was neutrality toward conclusion, to analyze methodology. I am not so sure why that would be hard to understand.

---

How does straightforward training and an expectation of neutrality toward conclusion make one immune to any particular kind of bias?

I'll deal with both of your replies here:1) If personal bias exists, then community bias can also exist. When you suggest that an individual can make an error in methodology due to desires about a conclusion, then so can a group of scientists that all hold the same bias. A biased peer review process will result in biased material getting through.Thus it is inconsistent for you to claim that creos must provide evidence of specific bad or biased reports to make the claim that there is bias, and then turn around and say you do not need to provide similar evidence when discussing bias.2) "How do you know you are immune to this kind of bias?" Very simple, I never focus on conclusions at all when going through a study. All I look at is methodology. One cannot determine methological flaws from the conclusion so I don't even bother. If the methodology proves flawed, severely enough that it is irritating, I may move on to conclusions or other parts of the study.That makes it pretty ironclad. If you abstain from dealing with X, its hard to make a mistake based on X... right?3) "How does straightforward training and an expectation of neutrality toward conclusion make one immune to any particular kind of bias?" Training itself will not, as I said in practice it depends on if you stick with your training. But otherwise this should be obvious. If training is to concentrate on objectivity and avoid a particular type of error by avoiding X, then by sticking to that training you will not have problems with X even if you slip sometimes on the objectivity.Your question seems absurd to me. Its like asking once I learned the scientific method, how can I claim that that instruction allowed me to follow the scientific method.4) Enough of your ad hominem bs. Let me now apply the same criteria you laid on the creos, if you are going to insinuate that I do have bias, please show me the evidence of it. I'll be very interested to see a quote (not mined) of mine stating that a study must be wrong because of its conclusion, of that in any way a conclusion indicates its methodology was wrong.The best you might get is one saying a conclusion makes me suspect methods may be wrong.This kind of work, careful objective analysis of methods and proper handling of data, was my work. In some cases it had to be well before results were known. Maybe that added to my ability to separate the two things in a practical sense.

---

holmes
"...what a fool believes he sees, no wise man has the power to reason away.."(D. Bros)

quote:

---

1) If personal bias exists, then community bias can also exist. When you suggest that an individual can make an error in methodology due to desires about a conclusion, then so can a group of scientists that all hold the same bias. A biased peer review process will result in biased material getting through.

---

Absolutely.But for an entire community of scientists to hold exactly the same bias for significant timespans is highly unlikely. Especially considering the competitive nature of science and the fact that careers are made by people overturning old paradigms.Also, someone claiming that the peer-review process is biased in a way that damages the credibility of the process would need to specifically demonstrate how it is so.

quote:

---

Thus it is inconsistent for you to claim that creos must provide evidence of specific bad or biased reports to make the claim that there is bias, and then turn around and say you do not need to provide similar evidence when discussing bias.

---

The thing is, I'm not talking about bias in scientific research and journals in the other thread.I'm talking about fraud or errors.

quote:

---

2) "How do you know you are immune to this kind of bias?" Very simple, I never focus on conclusions at all when going through a study.

---

"Never focus on" or "don't even know what they are?"

quote:

---

That makes it pretty ironclad. If you abstain from dealing with X, its hard to make a mistake based on X... right?

---

You claimed immunity to bias towards conclusions in a case where you already knew the conclusions, remember?

quote:

---

3) "How does straightforward training and an expectation of neutrality toward conclusion make one immune to any particular kind of bias?" Training itself will not, as I said in practice it depends on if you stick with your training.

---

Well, yeah."In practice", people often don't stick with their training.And by "often", I mean "frequently."Are you familiar with the work of Psychologists Kahneman and Tversky?They showed that training in logic and/or statistics doesn't help eliminate bias in many practical applications. There's another paper by them that I don't have online access to that goes into this issue in more detail. I'll see if I can get my hands on a copy and post the relevant parts in the near future. It's this one:Tversky, A., & Kahneman, D. (1983). Extensional vs. intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review, 293-3l5.

quote:

---

But otherwise this should be obvious. If training is to concentrate on objectivity and avoid a particular type of error by avoiding X, then by sticking to that training you will not have problems with X even if you slip sometimes on the objectivity.

​

Your question seems absurd to me. Its like asking once I learned the scientific method, how can I claim that that instruction allowed me to follow the scientific method.

---

The scientific method is based upon the assumption that individual bias cannot be completely eliminated.Otherwise, we wouldn't need peer review.

quote:

---

4) Enough of your ad hominem bs.

---

Huh?Look, you are the one who said you were immune to bias of this particular kind, not me. It was my mistake that I didn't keep focused on the particular kind of bias you mentioned, but you are still the one that made the claim about yourself.I'm just asking questions.

quote:

---

Let me now apply the same criteria you laid on the creos, if you are going to insinuate that I do have bias, please show me the evidence of it.

---

I am not saying that any specific example of your work is biased. How could I, since I do not have any examples of your work.I am saying that you are not immune to bias of any particular kind.No one is.

quote:

---

I'll be very interested to see a quote (not mined) of mine stating that a study must be wrong because of its conclusion, of that in any way a conclusion indicates its methodology was wrong.

---

But if you were biased, you wouldn't say that.That's why it's bias.Let me repeat that I'm not accusing you of any particular instance of bias, although you have already admitted that you are biased.I am just saying that you can't be immune to bias. Not any kind....because nobody is.

quote:

---

The best you might get is one saying a conclusion makes me suspect methods may be wrong.

​

This kind of work, careful objective analysis of methods and proper handling of data, was my work. In some cases it had to be well before results were known. Maybe that added to my ability to separate the two things in a practical sense.

---

You have already admitted that you are more willing to give papers whose conclusions you agree with more of a "free pass" than when you are suspicious of the conclusions (or the source) - which means you are likely to condemn "bad conclusions" for faults you pass over when the conclusions are "good".This is a classic example of being biased based upon conclusions, yet you said previously that it was not possible.Since I generally cite papers so that the conclusions are well-known to you in advance of you looking at the methodology of the study, how can you claim immunity to bias?

No its not. Look at the history of heliocentrism. What is letting something through, or disregarding a result, because of bias other than error or fraud? Perhaps I am making an equivocation and I don't realize it. Right now I do not see the distinction. Never focus on. Sometimes I do know what the conclusion is, especially if I read the synopsis first, or it was introduced to me by someone who mentioned the conclusion. Yeah, and I am still saying that. Just because you know the conclusion does not one iota mean you have to deal with it if you are instead dealing with the methodology.I'm not sure how this is not clear. Person A says to me, hey look at this study that says Bush is dumb. Great that seems to fit my experience, but now I look at what they use to determine this "dumbness". I may end up totally rejecting the study, despite agreeing with its conclusions, because the study is flawed. You really need to stop quoting studies. This has absolutely no bearing on what I am discussing.Look this is really simple. You go to a study see, and you ignore the conclusion part and the synopsis part, and every single part except for its methodology. You look at the methods and see what they did, as well as the discussion of how how the collected data logically leads to what they will conclude or fit in with other data.Methodology and logic. Those two guys are right that I might have a bias in trying to predict something, or in trying to carry out some chore. I already stated that I do have bias in how I might approach an article. What I do when I get my hands on it to review is something else. I look at procedural and logical errors. That is complete bs. The reason for peer review is because people can make mistakes. Yes its methods can catch people that are faking or biased, but that is not the assumption of peer review at all.Suddenly it looks to me like you have very little knowledge about science. Ad hominem bs. This began in the other thread when you decided to avoid dealing with actual arguments placed to you, by insinuating I could not be trusted because I claimed I was not liable to a certain type of error.You then opened up a whole thread on that specific subject. Tell me what is the purpose of this thread, that it could not have been handled in two posts in the other thread, or avoided altogether by you just dealing with my arguments, instead of claiming I couldn't be trusted? I am immune to bias of a specific kind, just as others can be. If I only criticize a study based on its methology, and refrain from criticism until I have seen the methodology, then I completely avoid the possibility of commiting the error of bias you suggest.If I don't ever drive cars, I cannot ever be a negligent car driver.If another person does not drive cars, they cannot be a negligent driver.If another person only criticizes methodology and refrains from criticism until having seen the methodology, then they avoid the bias you are talking about.Just as much as I or anyone else cannot accidentally get into a car, I or anyone else cannot accidentally base their criticism on something other than the methdology. Yeah, I said your claim that I said I never had bias was a big fat strawman. I then stated how I was biased. If I heard of a study that fits with my experience, I was less likely to review it in detail in the first place. Plus, if I caught errors I would likely only work up to the point that was necessary to say the study is a failure, and not continue further to point out how bad the author is.This is because if it agrees with me, I only need to know whether it in fact is something worthy of reference... the methodology is good or bad... and so I don't get caught out like a wanker as you often do because you do not fully read your refs.Now I have actually gotten caught, twice, being a complete wanker when I suggested something that was simply not real, because I assumed it was before actually reading the study. One of those times may have been at EvC. Can't remember.HOWEVER, your criticism was rejection bias. That a bias on my part against the conclusion of a study was likely to make me reject it. That is complete and utter bullshit.When something challenges my beliefs, I need to know whether it is solid or not, just like in the studies where I might agree with the conclusion. I then review the methology to find if it is good or bad. If it is good then that is it. There is nothing I can do about good data, accept to admit that it is good.If it is a bad study whose conclusions I do not like, then unlike a study whose conclusions I like but turned out to be bad so received necessary criticism, I may move on to some extra criticism. All valid, but unnecessary as the amount I had already found was enough to sink it.In other words I make the author of a bad article whose conclusions I do not like, sweat more, than those whose conclusions I might have liked.That does not in any way shape or form allow for the rejection of an article whose conclusions I do not like, because of the conclusion, rather than actual and factual methodological error. Being hard is different than being inaccurate. Because I look at the methodology. And when I cannot find the methodology then I do not comment beyond mentioning potential doubts. For proof of this you can refer back to the thread this all started in, where I specifically told you I could not deal with at least one study you mentioned because I had no access to it.

---

holmes
"...what a fool believes he sees, no wise man has the power to reason away.."(D. Bros)

quote:

---

No its not. Look at the history of heliocentrism.

---

Excuse me?Geocentrism was the prevailing notion before there was such a thing as professional scientists or peer review.There was no "scientific community" back then as I am using the term.So, my statement above holds.(By the way, I find it amusing that you are arguing that the entire scientific community can hold a major collective bias for an extended period of time, but that somehow, you, holmes, are immune. LOL!)

quote:

---

What is letting something through, or disregarding a result, because of bias other than error or fraud?

---

It would be bias, which would be error, but not fraud.Bias can't be fraud, because bias is unconscious.

quote:

---

Never focus on. Sometimes I do know what the conclusion is, especially if I read the synopsis first, or it was introduced to me by someone who mentioned the conclusion.

---

Then you are suceptible to bias.You cannot pretend that you never heard the conclusion once you have learned what it is.

quote:

---

Yeah, and I am still saying that. Just because you know the conclusion does not one iota mean you have to deal with it if you are instead dealing with the methodology.

---

But you cannot "unknow" the conclusions after you are aware of them, so you can be suceptible to bias.You know that old trick; "Stop thinking about white elephants."

quote:

---

Person A says to me, hey look at this study that says Bush is dumb. Great that seems to fit my experience, but now I look at what they use to determine this "dumbness". I may end up totally rejecting the study, despite agreeing with its conclusions, because the study is flawed.

---

But let's remember that this is not what you said. message 71

quote:

---

Well could it be that you are not understanding what is actually being stated?

​

Or could it be that certain groups of psychologists and psychiatrists have a stake in the capitalist "get healthy" goldmine which has a stake in everyone being victims?

---

message 72

quote:

---

Well could it be that you are not understanding what is actually being stated?

---

...to which you replied in Message 77:

quote:

---

NO, it cannot be that...

​

...I have the ability to disassociate myself from ramifications of a study, to clinically analyze its methodology and conclusions.

---

(I snipped the irrelevant derision.)You do not have the ability to "unknow" the conclusions of a study once you've read them.Stop thinking about white elephants.

quote:

---

You really need to stop quoting studies.

---

Yeah, I'll bet you don't much want to read these studies.

quote:

---

This has absolutely no bearing on what I am discussing.

---

What the studies show is that formal training in logic and statistics does not help to eliminate bias in many practical applications.You are using the fact that you have formal training in logic and statistics as support for your claim that you are immune from a particular type of bias in a practical application.How much more relevant do you need it to be?

quote:

---

Look this is really simple. You go to a study see, and you ignore the conclusion part and the synopsis part, and every single part except for its methodology.

---

Right.You make sure you never even see anything that might clue you in to the conclusion or even the subject matter. You are completely ignorant of those things.That will eliminate bias based upon conclusions for sure.But this is not what you said initially, remember?I posted some study results, you asked if it was possible that I didn't understand them, I asked if it was possible that you disagree with the studies because you didn't like the conclusions, and you said that no, this was not possible.We aren't talking about analysing a study, we are talking about your initial reactions to the little bits I posted in the thread.We damn sure weren't talking about you examining a study's methodology while being completely ignorant of the conclusions, because the conclusions were what you were reacting to.

quote:

---

Methodology and logic. Those two guys are right that I might have a bias in trying to predict something, or in trying to carry out some chore.

​

I already stated that I do have bias in how I might approach an article. What I do when I get my hands on it to review is something else. I look at procedural and logical errors.

---

Did you have your hands on any of the papers whose conclusions you rejected out of hand in the other thread?The claims you are making here seem quite unrelated to my original point.Let me restate what K & T found.Training in formal logic and statistics have been shown to not eliminate bias in practical applications which require logic and statistical knowledge.

quote:

---

The reason for peer review is because people can make mistakes. Yes its methods can catch people that are faking or biased, but that is not the assumption of peer review at all.

​

Suddenly it looks to me like you have very little knowledge about science.

---

Bias IS a mistake, your pedantic point aside.Errors often appear in research precisely because of bias.Why do you group "bias" with "fraud" above?Fraud is active lying; a conscious, knowing effort to deceive others. It is treated very, very harshly by the scientific community, as you know.Bias is, by definition, an unconscious tendency to see something in a particular way, which might be right or wrong. It is just something that all humans are very much prone to.It is not a human failing to be prone to it, nor is it something that one can always, or even often, conrol within oneself.Training can reduce it, but as Kahneman and Tversky showed, it does not eliminate bias in practical applications....like evaluating research studies as opposed to using formal logic constructions.The scientific community, through peer review,the practice of study replication, and the development of good research methodology, attempts to compensate for the multitude of human biases, because everyone has them, and they are not able to be controlled by sheer force of will.I completely agree that if a study evaluator is completely ignorant of a study's conclusions, subject matter, etc., and evaluates only the abstract methodology and is unable to glean any information at all about what the study was investigating, then conclusion bias is eliminated.But you cannot stop thinking about white elephants once someone tells you to.

You did not specify that it had to be exactly the same as today, so I was showing where a biased group did indeed control knowledge using the mechanisms they had at that time.If you want one today then I simply point you to my thread on the Rand study. The two top psychological/psychiatric organizations in the US (as well as the world) who have major control over studies (grants and publications) have publicly stated that sexual research (and one must assume all psych research) must fit current social and political beliefs. That will effect data and already has. I don't find it amusing in the least. I did not argue that I am a special case in being able to avoid ONE SPECIFIC TYPE OF BIAS. I argued that ONE TYPE OF BIAS CAN BE AVOIDED COMPLETELY. When groups (such as both APAs) refuse to remain objective and include the conclusion as part of when to criticize the accuracy of research, then THEY WILL MAKE THAT ONE SPECIFIC TYPE OF BIAS.It can happen and does happen. It is always when moralists, or "socialists", believe that consensus to current social or political paradigms are an equally important factor as methods in judging research. Well this seems like semantics. To my mind bias is bias. If one knowingly overlooks or presses false charges then it is fraud, if unknowingly it is error... in either case it is bias. How long do I have to endure this strawman? I did not suggest I was unsusceptible to all bias. I said I was unsusceptible to a specific kind. And indeed I am unsusceptible whether I know the conclusion or not. I didn't say "unknow" or mean to suggest "unknow". What I said is I can disassociate myself from the ramifications of a study to clinically analyze methodology. Perhaps it would help you understand what I mean by substituting "by clinically analysing" in place of "to clinically analyze"?Your white elephants are pink elephants. If one resorts to only critique of methodology, then one inherently removes the potential for bias you suggest. Scraf, I have read every study you have mistakenly presented me with. I have at each time shredded them, or in the case of very recently asked if you really need me to and giving suggestions of where I would head. At first I suggested you actually start reading and understanding studies before citing them inaccurately, or citing really bad studies.Since you refuse to do that for what reason I have no idea, I am now suggesting you simply drop citing studies. It almost universally ends up being a waste of my time. It appears I spend more time actually reading them than you do.You would be right that I don't want to read any more mistakenly cited, or truly bad studies. I would however love to see some correctly cited good studies which suggest what you say. It would only be interesting if it suggested you could commit errors of the type I was being accused of commiting, even if I used the technique I use. That people can make mistakes and have bias, does not mean that it can creep in everywhere. No, that's not what I said or meant. I said you ignore the conclusion to deal only with the methodology. That means not using the conclusion as a way to criticize or dismiss findings. I have already stated that I told you when I could not find the studies. I do not remember at this point if the question involved the ones I could find or not.I can tell you this though, I will thoroughly congratulate you on a nice skewer if you can show that those were studies whose methodologies I did not have some access to, and I actually rejected "out of hand". The clips of quotes you give don't even suggest that I am dismissing them, but rather that you are not understanding them, or that there might be a reason they could be wrong. That is different than saying they are definitely wrong.I asked for info on ones I could not get to, before I would actually state whether I agreed or not. Bias is a form of error, just as fraud is a form of bias which is a form of error. If we are simply talking past each other because of semantic issues, then we can accept yours for sake of argument in the future.In any case the scientific method was not devised nor in large part meant to handle "bias", or "error based on erroneous preconceptions". It is much more complex, and indeed deals also with positive issues too, but generally assumes objective activities by the researcher, correcting the commission of simple errors. A mathematical or mechanical error, or even an oversight has nothing inherently to do with "bias".But contrary to what you said, fraud may not just be accepted, it may even be congratulated by willfully ignorant people (groups of them in charge) for long periods of time. Witness the current APAs in action.

---

holmes
"...what a fool believes he sees, no wise man has the power to reason away.."(D. Bros)

From his perspective either the theistic view, the agnostic view or the atheistic view would be a blik. In other words I believe that we can never be truly objective.Here is a link to the entire item.
http://www.wellington-college.berks.sch.uk/...uslanguage.htmI think it is very pertinent to this discussion.Sorry, I don't know how to shorten the link.

---

Everybody is entitled to my opinion.

This is interesting but a bit off topic.What seems to be under discussion is someone who accepts a theory despite counter evidence, because of a desired conclusion. This is not the charge I was responding to.It is also about moving away from proper methodology. My discussion was sticking to proper methodology to avoid this sort of scenario, even if not the exact same sort of bias.It may be true that we can never be truly objective in our beliefs, but we can certainly stick to facts when analyzing the competency of a study and so be objective in analysis. It does take a choice to do this, but if made then the particular bias in question (rejecting a study merely because of its conclusion) is negated.

---

holmes
"...what a fool believes he sees, no wise man has the power to reason away.."(D. Bros)