Interview With Steven Novella of Skeptics Guide

[David Bailey]

Originally Posted by Paul C. Anagnostopoulos

How do you decide if they are changing the protocol to maximize the odds of failure or minimize the odds of leakage and cheating?

~~ Paul

I am not super keen on studying mediums to explore the existence of ψ - though if ψ is ever established, a study of what mediums do might be useful.

However, your question regarding protocols seems broader than that. For example, Weisman arbitrarily selected a protocol for the dog experiments, in which the first visit to the window was the only one to count - thus throwing away 99% of the video data collected. It is surely good experimental design to try to extract as much signal from the data as possible - unless the data is so voluminous that throwing most of it away doesn't matter.

David

[Paul C. Anagnostopoulos]

Originally Posted by Phronk

But in all seriousness...minimizing cheating and maximizing odds of finding a true effect should be exactly the same goal. A study where cheating is easy is useless to parapsychologists and a study where a signal couldn't get through even if it existed is useless to skeptics. If people just make it their goal to find the truth rather than getting all hot-headed about supporting what they already believe, a well-designed study shouldn't be hard to come up with.

I disagree. I think a well-designed study of psi is quite difficult. I sound like a broken record, but the reason is because the null hypothesis of almost every psi experiment is that no anomalous information was received by the subjects. The word anomalous is critical here: If the null hypothesis is discounted by a positive study, all it means is this:

The subjects obtained information via means that we do not understand.

Now, if everyone agrees that "do not understand" does not imply "spooky parapsychological explanation," then we're all set. But the way in which psi is discussed makes it clear that's not what most people assume. It's like studying UFOs: Most people, on both sides of the fence, use that term to mean "alien craft."

The problem with psi researchers bending over backward to make the protocol conducive to psi is that they are simply duplicating previous protocols that had positive results, even though they do not know why there were positive results. It is possible that they are simply duplicating leaks. They should be open to other researchers trying to close potential leaks, without lots of complaining about ruining the mood of the subjects. The reasonable psi researchers are.

~~ Paul

[Paul C. Anagnostopoulos]

Originally Posted by David

However, your question regarding protocols seems broader than that. For example, Weisman arbitrarily selected a protocol for the dog experiments, in which the first visit to the window was the only one to count - thus throwing away 99% of the video data collected. It is surely good experimental design to try to extract as much signal from the data as possible - unless the data is so voluminous that throwing most of it away doesn't matter.

Yeah, the whole dog thing is a problem. I've seen videos and I suspect it's typical that a dog will wander about all day, sometimes going to a window, sometimes a door, sometimes his water bowl. Which one of the visits to the window or door is the psychic one? You have to make sure that the owner leaves for home at a random time, then calculate the probability that the dog goes to the window at that time. It's a probability calculation since the dog goes to the window more than once, and because there is some chance that the dog goes to the window more often as the day goes by. Also, you have to agree beforehand when the window visit counts: when the owner is informed that it's time to leave for home, when he gets up to go, when he gets into his car, when he leaves the parking lot, etc. And certainly you cannot count any time when the car is within earshot of the dog.

Let's say we agree that it's when the owner gets into his car. One way to tighten up the experiment would be to have him get into his car, drive around the parking lot for a minute, then go back to work. That eliminates any question of the dog hearing the car.

Do we use a judge to decide whether the dog went to the window in the agreed-upon timeframe? Or do we simply require a visit within a 1-minute period or something like that?

~~ Paul

[Open Mind]

Well, 'skeptics' (i.e. confident disbelievers) for years accused parapsychologists of a 'file drawer effects' – only publishing the successful studies, which is subtle way of accusing trained parapsychologists of selective reporting fraud. If they believe mediums and parapsychologists so readily cheat, well surely one must also assume that a sceptic with a political stance against paranormal claims would bend the results to prevent embarrassment? What is the difference?

I mean I don't see glowing honesty and neutrality amongst the paranormal debunkers. Randi counts applications (non pursued tests) as failed tests. He once said JREF has tested Dog ESP, they hadn't (false memory syndrome? ) In one UK TV program Randi reportedly hot read after his cold reading attempt had been clearly unsuccessful (he was on a TV program for comparison with medium).

CSICOP's 'starbaby', where co-founder Dennis Rawlins resigned after his results of a test, which embarrassingly to CSICOP didn't support their stance against astrology (not that I believe in astrology without far better evidence either but that is not the point)

I also have spoken to debunker magicians in past, enquiring about the possibility of the magicians doing cold reading VS mediums doing their thing, one magician refused only to cold read when asked about the possibility of signing a statement not to hot read, he said given the opportunity he would definitely attempt to hot read because he believed mediums hot read. He only wanted to beat the mediums. If I hadn't mentioned the possibility of signing a statement, I am not convinced he would have acknowledged his intentions to cheat at all, it could havebeen labelled cold reading.

How can one insure that a medium who is mildly successful in such a test, isn't automatically dismissed as a failure? Demkina was dismissed as failure with Hyman saying she should give it up, after she achieved odds of over 50 to 1 in her favour.

Similarly Novella comments on the failure of a coin tosser. '.....
One final testament to the coin-tosser’s inability to grapple with statistics— he was somewhat disappointed with the fact that Randi would not award him the 1 million dollar prize for proving his abilities to a P-value of 0.05 (which basically means that the probability of the results being due to chance alone is 5%). If that’s good enough for medical research, he argued, then why wouldn’t it be good enough to prove his abilities. I tried to explain to him that Randi has no intention of handing 1 million dollars to every 20th person (on average) who takes the challenge. He didn't get it (literally and figuratively).


The good thing here is that the bar setting for 1 million prize is removed so it is not so easy to justify dismissing a mild success, as a definite failure. A mild success by medium could be used to force the sceptics to do further research, longer trials? Perhaps I am being too optimistic, Hyman's response after Demkina achieving 50 to 1 was that he hoped she would give it up and go do something else worthwhile with her life. Skolnick reported the ' the girl with the very ordinary eyes' .

I am not saying Demkina was necessarily psychic but what was astonishing was to see the mentality of CSICOP (CSI) at work, a unbreakable confidence that whatever was occurring it was certainly not psi and they had no scientific curiosity to test further or prove alternate explanations. They didn't just express reasonable doubt as in 'there is not sufficient evidence in a one off trial' but actually expressed it more like a claim of failure without taking the burden of proof that should accompany any claim or counter claim.

[Paul C. Anagnostopoulos]

Originally Posted by Open Mind

Well, 'skeptics' (i.e. confident disbelievers) for years accused parapsychologists of a 'file drawer effects' – only publishing the successful studies, which is subtle way of accusing trained parapsychologists of selective reporting fraud. If they believe mediums and parapsychologists so readily cheat, well surely one must also assume that a sceptic with a political stance against paranormal claims would bend the results to prevent embarrassment? What is the difference?

There is no difference. I'm sure there is a background level of cheating, both consciousness and nonconscious, in every field of endeavor. But we agree it's not fair to throw the baby out with the bathwater, right? Well, except in Uri Geller's case, where he should have been tossed long ago.

CSICOP's 'starbaby', where co-founder Dennis Rawlins resigned after his results of a test, which embarrassingly to CSICOP didn't support their stance against astrology (not that I believe in astrology without far better evidence either but that is not the point)

The Starbaby saga is such a he-said she-said morass that I completely ignore it. I have no idea what happened nor whom to trust.

I am not saying Demkina was necessarily psychic but what was astonishing was to see the mentality of CSICOP (CSI) at work, a unbreakable confidence that whatever was occurring it was certainly not psi and they had no scientific curiosity to test further or prove alternate explanations.

Sorry, neither would I. She wasn't seeing into people's bodies. And what the hell were her parents doing? Child abuse.

You have to realize that after looking at a lot of these claims, one becomes cynical. That's not an excuse, but it's there.

~~ Paul

[Open Mind]

Originally Posted by Paul C. Anagnostopoulos

And what the hell were her parents doing? Child abuse.

What do you mean by this? I've not looked at this case for years, what happened?

[uberdoubter]

To be honest, I am a regular listener to the SGU. Someone said they would like to hear what a fan of the SGU thought of the interview. I thought the SGU guys were unwaiveringly patient and kind with Alex. Steve explained again and again why the data for doggy psi and psi in general is not compelling. Alex would respond with canards like "That isn't my cup of tea" and That doesn't interest me." and ask to move on to the next topic. He repeatedly said good data is out there, but when cornered on it, he made it sound unimportant. So which is it?
Also, most here in this forum missed the point. Over and over, test after test, positive results for psi shrink to noise as better methods are used. You can make any excuse you like, but you are reduced to anecdotes as evidence.

[Paul C. Anagnostopoulos]

Originally Posted by Open Mind

What do you mean by this? I've not looked at this case for years, what happened?

Sorry, didn't mean to get you excited. I think parading a kid around claiming she can see inside people's bodies is a form of child abuse. Rather reminds me of the disgusting Audrey Santo case.

~~ Paul

[DysonSphere]

Originally Posted by uberdoubter

Also, most here in this forum missed the point. Over and over, test after test, positive results for psi shrink to noise as better methods are used. You can make any excuse you like, but you are reduced to anecdotes as evidence.

No, the positive results get smaller as the designs get more complex, so they could to test to see how psi works, the controls were tight throughout. In fact many of the complex designs in the ganzfeild that used musical targets instead of visual targets and such were predicted by the researchers to get null results.

I'm sick of armchair critics. Skeptic should collaborate with parapsychologists to settle the issue instead of attacking their work from the sidelines.

[Phronk]

Originally Posted by Paul C. Anagnostopoulos

I disagree. I think a well-designed study of psi is quite difficult. I sound like a broken record, but the reason is because the null hypothesis of almost every psi experiment is that no anomalous information was received by the subjects. The word anomalous is critical here: If the null hypothesis is discounted by a positive study, all it means is this:

The subjects obtained information via means that we do not understand.

Now, if everyone agrees that "do not understand" does not imply "spooky parapsychological explanation," then we're all set.

I agree completely that the negative definition of psi is a problem. But I also think that demonstrating conclusively that information can be transferred by means other than the currently known senses (and I don't include just the five of them there), and/or against the normal flow of time, that's pretty damn important. Even if we don't know exactly what the "means" are, the fact that they are not any of the means we already know about means they are undiscovered scientific territory that should get any real scientist excited.

But the thing is, since we only need to rule out things we already do know about, it shouldn't be hard. We just take the means we know about, block them from operating, and see what happens. And I don't mean to imply that it's easy, but with enough intelligent people from enough disciplines devoting enough time to it, a good psi experiment is possible. In my opinion, a good body of work has already come out of it, but the "enough" part has been lacking.

The problem with psi researchers bending over backward to make the protocol conducive to psi is that they are simply duplicating previous protocols that had positive results, even though they do not know why there were positive results. It is possible that they are simply duplicating leaks. They should be open to other researchers trying to close potential leaks, without lots of complaining about ruining the mood of the subjects. The reasonable psi researchers are.

I agree again. I always find the "psi conducive atmosphere" excuse to be premature at best. Besides, there is no reason why plugging leaks absolutely has to affect the mood of subject negatively. If a subject wants to be useful, they should be happy to participate in a leak-free study.

For a concrete example, I think it is absolutely necessary that, when trying to replicate Sheldrake's work, the "going to the window more and more" explanation is completely ruled out. I think Alex is not taking this problem seriously enough. It's better to rule it out in the protocol than to try to explain it away post-hoc (as Sheldrake has done, and done well, but it's still post hoc). It's not enough to point out the 4% vs. 50% statistic - if the dog could sometimes rationally deduce when the owner was coming home due to this problem, and boost up their time at the window when they can, they could very well make the gap in percentages quite large. A small leak can lead to large numbers.

I've posted more about this on Open Source Science.

I've wandered off topic...but my main point is that all of this can be accomplished if everyone involved puts in a little intellectually honest effort.