One for the skeptics?

[David Bailey]

Rudism,

The problem is that I suspect a different experimental design would attract sceptical fire too. Suppose that each trial had exactly the same number of arousing images. As I understand it, this would eliminate the statistical flaw we are discussing (it really needs a name!) but such experiments would then be open to the accusation that the subjects could exploit the less random nature of the presentation sequence.

David

[Rudism]

Well, I didn't say it would be easy to come up with such a design, only that it would be much more compelling if someone were to do so and then still get positive results. Statistics can be messy, and you can often come to completely different conclusions using the exact same data depending on how you actually select your computational set and how you do the math (this also ties in to problems with meta-analyses that have been discussed in other threads on the board).

The simplest way I can think of would be to only run each participant through a single trial, without warning them of the upcoming response-evoking stimuli. For example, tell the participants that you are simply measuring how relaxing classical music affects skin conductance, and put a loud shocking bang somewhere in the track. Get a baseline measurement initially, then measure their conductance immediately leading up to and after the bang. Apologize (say it was an unforseen technical difficulty), and then send him off and move on to the next participant.

Problems I could see with this would be that it may be tough to get by an ethics committee, and it would require a significantly higher number of participants to get the same number of trials (you would also have to control participants from communicating with each other about their experiences, otherwise people could clue into what you're doing and expectancy could creep in even here).

Anyone else have any ideas about how expectancy could be completely ruled out at the experimental design phase?

[David Bailey]

Rudism,

Perhaps you are beginning to see my point - attempts at total rigour stifle any experiment - PSI or otherwise. I would certainly be convinced if the various trials were a 50/50 mixture of the traditional totally random type and trials with a guaranteed number of events, and the results for the two sets of data were broadly similar.

I must admit, having read all the above, I am becoming cautiously optimistic that presentiment is real - because if it is, it should (eventually!) give us some fascinating new science.

David

[mcromer]

Rudism,

Looking at their data analysis, the expectation effect simply cannot work.

That is because the timing of the stimulus is random. There is no way for anyone or anything to know when the next stimulus is going to occur, absent psi.

Look specifically at these lines:

We now have the following three conditions: An excess of skin conductance responses in the prestimulus epoch prior to audio stimuli, no such excess over chance expectation in the preceding epochs and a randomized inter-stimulus interval of 40 to 80 seconds. Any expectation effect must appear as an increase in the dependent variable (i.e., rate of production of SCRs), which is a monotonically increasing function of the time since the last arousing stimulus. Such an effect, were it to occur, could not give rise to an increase in the SCR rate solely in the epoch prior to the next startle stimulus, a fortiori when the timing of that stimulus was randomly varied and unknown to the participant.

The next couple of paragraphs seal the deal -- expectation effects simply cannot account for the experimental results in this paper.

However, I "expect" I will continue to hear positional materialists (mislabeled as "skeptics", however I never hear an ounce of skepticism from them about supposed materialist explanations of reported psi phenomena) harping on how the presentiment studies can be fully explained as a simple statistical artifact of expectation effects for years and decades to come.

Keep in mind that the authors themselves hypothesize that these results could be a result of DAT (essentially experimenter psi), and there have been failed replications of presentiment effects (although lots and lots of successes, too). I think the best argument for the positional materialists at this juncture is simply to argue for the conspiracy theory that the parapsychologists who are successful at this experiment are all frauds.

[Topher Cooper]

No one, including Dean Radin, would not say that it wouldn't have been nice to exclude expectancy from the results a priori, whether by controls or by analysis (it is routine, of course, for a design to be such that alternatives are excluded by analysis). This is the case with later presentiment experiments and yet, somehow, though the Skeptics claim to reject this experiment because this was not done in the original, they are not enthusiastically embracing the experiments for which it does not hold.

You will note that I always refer to these people as Skeptics with a capital "S" or occasionally as pseudo-skeptics but never as skeptics small "s". Any issue of Skeptical Inquirer could stand, for example, as a textbook of logical fallacies and rhetorical dirty tricks -- and not by pointing out those flaws in others.

[alextsakiris]

Would any of you be interesting in helping me produce a Skeptiko episode dedicated to this experiment?

I'd like to really dig into it and interview several of the researchers involved as well as Dean Radin.

Any thoughts on this?

[Topher Cooper]

Alas, my vacation ends tomorrow and I doubt I'll have much free time. I would definitely recommend including Dick Bierman (his English is excellent so that will present no problem). You might consider including someone expert in neuro-physiological measurements who is completely neutral (preferably new to the controversy). Not my field, so I wouldn't know who to suggest -- perhaps one of the researchers who performed the existing conventional studies in which Dick found presentiment data in.

[mcromer]

Alex,

Let me know what you need and I'll be glad to help out.

[David Bailey]

Alextsakiris,

I think devoting a Skeptiko episode devoted to this effect would be fantastic!

I don't feel qualified to help as such, but here is a suggestion for everyone to knock about:

You once told me you were considering trying the presentiment experiment as part of OpenSourceScience. This would be a great opportunity to challenge the Sceptics. Perhaps you should give them advance notice of the following question:

Can you devise a protocol to test the presentiment effect that would satisfy you that the result - if positive - was not due to the expectation effect? If so, you yourself could use it for the experiment! The experimental design would have to be reasonable - i.e. take into account that subjects could only tolerate trials of a certain length before boredom set in.

Alternatively, do they wish to state that it is inherently impossible to test this effect scientifically - even if it is genuine!

This should make a fascinating podcast - particularly if Dean Radin and Dick Bierman take part!

David

[jacob]

I also think this would be a great subject for an episode and have a live discussion between Dean Radin and others would be a good way to come to some conclusion fast (meaning during the show).

Especially so, since this experiment seems to be such an important part of the psi supporters/skeptics debate and since it is so good by using a simple and repeatable procedure.