Naturalism Advocate Sees No Evidence for Survival of Consciousness After Death (Podca

[mszlazak]

It's not a myth that psi effects diminish or disappear when possible non-psi reasons are found and explored. A good example is the Ganzfeld studies. They initially started from a larger point until after some of the suggestions in the Hyman-Honorton joint communication were implemented and Bem's first meta-analysis. They decreased to zero after the Milton/Wiseman meta-analysis. Recovered slightly with the subsequent exchange but not to the levels prior Milton/Wiseman and ended at a lower level with the last Broughton/Bem article on the subject in their exploratory paper.

Now we're finding out that a similar thing is going on with the presentiment experiments. History repeating itself and quiet consistently when it comes to parapsychology. Every time some error is found supposed "parapsychology effects" trend toward zero. So skeptics and scientists just see a pattern of diminishing returns with the elimination of "error."

Worst. This lack of results in parapsychology is held within an framework without a theory about psi. This framework is itself a logical fallacy: the eliminativist nature of psi's research program (as mentioned in my previous posts). It is a naive Sherlock Holmes type eliminative induction that is warned about philosophy-of-science textbooks. Ronald Giere's 1980's text as one example:

If the original alternatives are definite enough and few enough, and one can be sure that they are all there, then one stands some chance of being able to eliminate all but one. This is almost never the case when the alternatives are THEORETICAL HYPOTHESES about some complex system. For a complex system there will be infinitely many different possible hypotheses, only one of which is true. Rarely would all the possible hypotheses be so neatly ordered that one could in some way eliminate all but one. Usually, no matter how many possibilities one succeeds in eliminating, there are still infinitely many left. It is impossible to get down to only one.(1984, p.170)

Moreover, philosopher of science, John Earman says that these criticisms apply even more so to eliminative inductions with observational generalizations (which is what mostly parapsychology experiments are). They cannot succeed.

So real scientists or people that understand the problem, are holding out on methodological grounds for stable enough effects and a theory of psi with specific testable consequences.

Until then, there is no convincing evidence for psi.

Some other alternatives, suggested by observers of the field, that could explain pseudo-psi effects and give parapsychologists something to contribute, is more knowledge on implicit learning and expectancy effects.

----------------------------

[Topher Cooper]

There is a tendency, in all fields, for experimental effect sizes to diminish over time. It used to be something of a joke that the speed of light appeared to decrease over time earlier experiments showing a larger value than later. This tendency is rather mysterious in physics, but is actually quite understandable in fields where human performance (and thus enthusiasm) is important.

There was certainly a decrease in effect size in the Ganzfeld. By careful selection of small studies and an arbitrary cut-off date that excluded some positive studies, as well as a change in test criteria they were able to show that the additional studies they looked at -- although their effect size was only somewhat smaller than the older studies -- were not significant (i.e., were consistent with no effect). These studies were not, however, of any better quality than the earlier studies (Wiseman and Milton didn't claim that they were). Many of the studies were process oriented and included conditions that were predicted by the experimenters to produce poorer response. Others experimented with variations on the basic conditions (e.g., using music rather than images as the target).

When all the studies, including the earlier studies and studies ignored by Wiseman-Milton were included, a robust effect was nevertheless obtained. When studies were restricted to those which used the original basic design then the results remained substantially the same.

You cannot simply declare that lower effect size must be due to better controls and therefore conclude that better controls produce lower effect sizes. This is circular reasoning and is all too common in the Skeptical literature.

And what we are finding with the presentiment literature is that there was a flaw in the very first experiment that was corrected and seemed to have no influence on the outcome. No decrease shown with increasing controls -- just the usual unsupported claim.

[mszlazak]

Originally Posted by Topher Cooper

There is a tendency, in all fields, for experimental effect sizes to diminish over time. It used to be something of a joke that the speed of light appeared to decrease over time earlier experiments showing a larger value than later. This tendency is rather mysterious in physics, but is actually quite understandable in fields where human performance (and thus enthusiasm) is important.

There was certainly a decrease in effect size in the Ganzfeld. By careful selection of small studies and an arbitrary cut-off date that excluded some positive studies, as well as a change in test criteria they were able to show that the additional studies they looked at -- although their effect size was only somewhat smaller than the older studies -- were not significant (i.e., were consistent with no effect). These studies were not, however, of any better quality than the earlier studies (Wiseman and Milton didn't claim that they were). Many of the studies were process oriented and included conditions that were predicted by the experimenters to produce poorer response. Others experimented with variations on the basic conditions (e.g., using music rather than images as the target).

When all the studies, including the earlier studies and studies ignored by Wiseman-Milton were included, a robust effect was nevertheless obtained. When studies were restricted to those which used the original basic design then the results remained substantially the same.

You cannot simply declare that lower effect size must be due to better controls and therefore conclude that better controls produce lower effect sizes. This is circular reasoning and is all too common in the Skeptical literature.

And what we are finding with the presentiment literature is that there was a flaw in the very first experiment that was corrected and seemed to have no influence on the outcome. No decrease shown with increasing controls -- just the usual unsupported claim.

There maybe a tendency for some effects to diminish in all fields over time but the reasons for this are understood. So what?

Anyway, I do need to change my emphasis a bit. Meta-analysis is a post hoc data analysis not for corroberation or confirmation but for exploration since it is a poor substitute for one large well-conducted trial. Furthermore, the meta-analyses done in parapsychology are of poor quality and don't even make it to the level of those done in pharmaceutical research.

This is the current thinking on meta-analysis in science that has been gained from years of experience working with them: In recent years, it's got to the point where the conclusions of a meta-analysis are typically accepted only to the extent that they are supported by statistically significant results from large well-designed studies.

Parapsychology is "behind the times" in the quality of their meta-analyses and wrong about what they can be used for.

As you last post indicates, this post hoc data analysis suffers from biases that effect the analysis. I can provide more evidence of what your post illustrates in parapsychology but suffice it to say that in parapsychology there are serious controveries over alternative methods, criteria, and outcomes with regards to meta-analysis. Also, unlike what's going on in say pharmaceutical research, there is no well-designed ganzfeld study with a pre-planned power analysis to allow one to gather some exploratory results from any ganzfeld meta-analyses.

Moreover, the variablity among these parapsychology studies combined with the negative correlation between effect size and sample size raises more doubts about these meta-analyses because this is normally diagnostic of bias in a meta-analysis, [Eggar, M., Smith, G.D., Schneidler, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple graphical test. British Medical Journal, 315, 629-634.].

So, there isn't any psi evidence worth considering.

However, say there was some consistent evidence coming from parapsychology that is worth considering. Then a much more serious problem remains. What is it evidence of? Some type of "normal" effect(s) of which there could be many, systematic error(s) or maybe just maybe psi. With no theory of psi you are left with the logical fallacy I mentioned before that can be associated with eliminative induction. Also, since there is no theory of psi then there is no specific deductive consequences that can be falsified. Psi remains as just some unfalsifiable belief.

----------------------

[mcromer]

So, there isn't any psi evidence worth considering.

Veridical NDEs, the thousands of case studies of psi phenomena like crisis apparitions, an almost infinite number of reports of precognition, studies with astronomically significant odds against chance.

All dismissable in an instant by you. . .

Whatever.

[Topher Cooper]

Let's look at the proposed method of meta-analysis (as I said, whenever you deal with multiple experiments, you are performing a meta-analysis -- however badly).

First lets look at a similar method:

Select the largest study from among those that are well conducted. Throw out all other data.

I'm going to assume that this was done correctly. That the evaluation of the set of well-conducted studies was done blind to the outcome and that the proposal to use this method was done blind to the actual content of the set of studies. Unlikely, but let's assume it.

Then this is a classic example of one of the two problems that are typical of how the Skeptics like to evaluate evidence for things that they disapprove of -- use methods that weaken the strength of the evidence. In this case, its pretty direct, you start by throwing away much of the evidence that you don't like.

There is a odd idea going through much Skeptical argument. That is that smaller studies are more prone to producing false positives (i.e., producing a positive outcome when the reality isn't actually positive) than larger studies. This isn't the case at all. A fundamental property of statistical analysis is that the rate of false positives is preset and independent of the study size. What is actually the case is that smaller studies are more prone to false negative, i.e., failing to find an effect that is actually there.

Where did this idea come from. Well there is a tendency (more so in some fields, such a medicine than in others) for researchers to conduct small, sloppy experiments just to have conducted the experiment. In those cases we have experiments that are both small and sloppy -- but not sloppy because they are small. In those fields where this is rife there are reasonable warnings to be particular careful about small studies. But it is not valid to assume that a study is improperly done, even in those fields, just because it is small.

A small study that is carefully done and produces a positive outcome is equal in evidence to a large study that is carefully done and produces a positive outcome.

In fact I would rather have ten studies of 100 samples each than a single large sample of 1000 samples. The reason, any individual experiment may have some subtle, unrealized flaw in how it was conducted (impure reagents, for example, in a chemistry experiment), but this is unlikely in independent replications. This is why replication is considered an important part of science.

Part of a proper, formal meta-analysis is to take into account in the evaluation the effects of potential flaws, small or large. I have nothing against larger studies (when the resources are available to conduct them) but don't throw out most of your data -- you have an even better result if you include the large studies in your meta-analysis with the smaller ones.

But, of course, that is not what is proposed here.

What is being proposed is to first throw out the existing studies, which have produced the "wrong" results and do a "proper" large experiments.

This as well as being weak is a biased procedure. If you throw out results that don't fit your preconceptions, you have a clear bias, as well as producing a weaker result.

Of course, this was done. Charles Honorton decided to take up the challenge -- however unjustified -- of Skeptics. He set out to design a version of Ganzfeld that met all the criticisms presented by Skeptics, a design referred to as the Auto-Ganzfeld, and to conduct a large study with it. He consulted with arch-Skeptic Ray Hyman, as well as many more neutral outsiders, to create a design that would be beyond criticism. One of the leading experts in experimental psychology was consulted and declared that if the experiment produced positive results it would be strong unambiguous evidence that psi exists.

A large study was conducted which produced a quite strong result and massive evidence for psi. The expert, Daryl Bem, supported the results to the extent that he placed his own professional reputation on the line by co-authoring the resulting paper in a major psychology journal.

Ray Hyman refused to acknowledge the success even though he had signed off on the design in advance. He cited a number of rather bizarre new flaws. These were eliminated from the design and the experiment run again producing nearly identical results.

Finally even Hyman couldn't think of anything else that might have been wrong with the design. He reluctantly wrote a statement to the effect that the experiment appeared perfect in design and execution -- but that that did not mean that it proved anything. There might be unknown flaws in the design that explained the results.

The experiment was successfully replicated (and, of course, it was a replication, with more elaborate controls and a larger size, of previous experiments). Still, no acknowledgment that the large, clear-cut, with excessive controls in place that had been demanded as being all that was needed, had, in fact been done.

Instead we get the argument that single experiments don't count. What is really needed is that a large number of experiments, many of them quite differently conducted, and selected by Skeptics, using different methods of analysis agree with the outcome in detail. If meta-analysis produces the "wrong" result, demand a single large experiment done to the Skeptics specification. When that produces the wrong result, demand a meta-analysis done in such a way as to make positive results unlikely.

Of course, we've seen this dispute in this very thread.

Pardon me if I am less than impressed by such "objective" requests.

[daresh]

Originally Posted by mszlazak

It's not a myth that psi effects diminish or disappear when possible non-psi reasons are found and explored. A good example is the Ganzfeld studies. They initially started from a larger point until after some of the suggestions in the Hyman-Honorton joint communication were implemented and Bem's first meta-analysis. They decreased to zero after the Milton/Wiseman meta-analysis. Recovered slightly with the subsequent exchange but not to the levels prior Milton/Wiseman and ended at a lower level with the last Broughton/Bem article on the subject in their exploratory paper.

Hmm the milton/wiseman meta analysis is itself very controversial.
The results from the hyman-honorton research can't be ignored even with the wiseman analysis...

anyhow you seem to have had this discussion already on the comments of dean radin his blog Entangled Minds: Some noteworthy books
where you said the same arguments of the diminishing results but this doesn't seem to be true according to Dean Radin and you didn't reply anymore to his argument...
Anyhow
I can't wait till I get my sociology degree so I can check these things out myself without relying on other people. My personal experiences with telepathy showed me that it exists but ofcourse that's subjective

An intresting comment from Dean Radin:
"My initial reading of the psi literature many years ago piqued my interest in this field, but it wasn't until I was actually running my own experiments for about five years that I began to find the null hypothesis inadequate. In hindsight then, I expect that for those who start out skeptical of psi (which is probably the default for many scientifically trained folks), I doubt that just reading the literature would ever change your mind."

I think this is true for those who are skeptic but still open minded
Anyhow Wiseman is a person who isn't open minded at all, remember the Demkina Case? Scientists use Media for Propaganda
That's an example of sloppy science.
Mark have you done psi experiments yourself?

greets,
Filip

[mszlazak]

Originally Posted by Topher Cooper

Let's look at the proposed method of meta-analysis (as I said, whenever you deal with multiple experiments, you are performing a meta-analysis -- however badly).

First lets look at a similar method:



I'm going to assume that this was done correctly. That the evaluation of the set of well-conducted studies was done blind to the outcome and that the proposal to use this method was done blind to the actual content of the set of studies. Unlikely, but let's assume it.

Then this is a classic example of one of the two problems that are typical of how the Skeptics like to evaluate evidence for things that they disapprove of -- use methods that weaken the strength of the evidence. In this case, its pretty direct, you start by throwing away much of the evidence that you don't like.

There is a odd idea going through much Skeptical argument. That is that smaller studies are more prone to producing false positives (i.e., producing a positive outcome when the reality isn't actually positive) than larger studies. This isn't the case at all. A fundamental property of statistical analysis is that the rate of false positives is preset and independent of the study size. What is actually the case is that smaller studies are more prone to false negative, i.e., failing to find an effect that is actually there.

Where did this idea come from. Well there is a tendency (more so in some fields, such a medicine than in others) for researchers to conduct small, sloppy experiments just to have conducted the experiment. In those cases we have experiments that are both small and sloppy -- but not sloppy because they are small. In those fields where this is rife there are reasonable warnings to be particular careful about small studies. But it is not valid to assume that a study is improperly done, even in those fields, just because it is small.

A small study that is carefully done and produces a positive outcome is equal in evidence to a large study that is carefully done and produces a positive outcome.

In fact I would rather have ten studies of 100 samples each than a single large sample of 1000 samples. The reason, any individual experiment may have some subtle, unrealized flaw in how it was conducted (impure reagents, for example, in a chemistry experiment), but this is unlikely in independent replications. This is why replication is considered an important part of science.

Part of a proper, formal meta-analysis is to take into account in the evaluation the effects of potential flaws, small or large. I have nothing against larger studies (when the resources are available to conduct them) but don't throw out most of your data -- you have an even better result if you include the large studies in your meta-analysis with the smaller ones.

But, of course, that is not what is proposed here.

What is being proposed is to first throw out the existing studies, which have produced the "wrong" results and do a "proper" large experiments.

This as well as being weak is a biased procedure. If you throw out results that don't fit your preconceptions, you have a clear bias, as well as producing a weaker result.

Of course, this was done. Charles Honorton decided to take up the challenge -- however unjustified -- of Skeptics. He set out to design a version of Ganzfeld that met all the criticisms presented by Skeptics, a design referred to as the Auto-Ganzfeld, and to conduct a large study with it. He consulted with arch-Skeptic Ray Hyman, as well as many more neutral outsiders, to create a design that would be beyond criticism. One of the leading experts in experimental psychology was consulted and declared that if the experiment produced positive results it would be strong unambiguous evidence that psi exists.

A large study was conducted which produced a quite strong result and massive evidence for psi. The expert, Daryl Bem, supported the results to the extent that he placed his own professional reputation on the line by co-authoring the resulting paper in a major psychology journal.

Ray Hyman refused to acknowledge the success even though he had signed off on the design in advance. He cited a number of rather bizarre new flaws. These were eliminated from the design and the experiment run again producing nearly identical results.

Finally even Hyman couldn't think of anything else that might have been wrong with the design. He reluctantly wrote a statement to the effect that the experiment appeared perfect in design and execution -- but that that did not mean that it proved anything. There might be unknown flaws in the design that explained the results.

The experiment was successfully replicated (and, of course, it was a replication, with more elaborate controls and a larger size, of previous experiments). Still, no acknowledgment that the large, clear-cut, with excessive controls in place that had been demanded as being all that was needed, had, in fact been done.

Instead we get the argument that single experiments don't count. What is really needed is that a large number of experiments, many of them quite differently conducted, and selected by Skeptics, using different methods of analysis agree with the outcome in detail. If meta-analysis produces the "wrong" result, demand a single large experiment done to the Skeptics specification. When that produces the wrong result, demand a meta-analysis done in such a way as to make positive results unlikely.

Of course, we've seen this dispute in this very thread.

Pardon me if I am less than impressed by such "objective" requests.

Hey Topher, forget the history lesson!

The state-of-the-art thinking about meta-analysis is it CANNOT be used for corroberation or confirmation. They are only exploratory in the best circumstances.

That means that all the meta-analyses done in parapsychology are at this point worthless.

The individual studies, individual experiments, the went into these analyses don't have enough power so any post hoc techniques can't save them.

All these experiments show is that they can implimented to take into account various criticisms and concerns. They are basically "pilot studies."

Once again. Meta-analysis CANNOT be used for confirmation or corroberation. This is knowledge that has been in the textbooks for a few years [e.g: Green, S., Benedetti, J., & Crowley, J. (2003). Clinical Trials in Oncology, 2nd ed. p. 231]. Parapsychology researcher Richard Wiseman emailed me back saying he thought the most recent Broughton & Bem meta-analysis paper on Ganzfeld was "exploratory." Given what is know about the problems with meta-analysis, how could it be anything else!

So this isn't thinking coming from a small coterie of loud so-called "skeptics."

It's the thinking in mainstream science.

Moreover Topher. It's the thinking that has recently started to run through parapsychology. Caroline Watt, in her 2005 presidential address to the Parapsychology Association raised this issue but didn't entirely get it right. However, the mess that is meta-analysis has risen to the conscious awareness of parapsychology. The paper of J.E. Kennedy (2004), "A Proposal and Challenge For Proponents and Skeptics of Psi." Jouranal of Parapsychology, 59, 47-62, was specifically mentioned by Watt. Some errors in Watt's talk will be addresses in an upcoming letter to the editor by J.E. Kennedy currently (08-05-2007) called "Letter on Meta-Analysis in Parapsychology."

Forgetting all this meta-analysis stuff, is there yet a specific fruitful theory of psi which can be falsified or are we still dealing with the logical fallacies of naive eliminative induction?

[David Bailey]

I know this is just a blog, but it seems to have typified what has gone wrong with this corner of science.

The 'sceptics' seem to feel it would deal a fatal blow to science if they ever admitted that any PSI result seemed valid - or even interesting. What they never seem to acknowledge is that if this sort 'rigour' were applied to the rest of science, the entire process would halt. For example, how much research would get done if everyone had to prove that they had not cheated?

Paradoxically, if PSI really doesn't exist, the sceptics seem to be endlessly postponing the day when this will be generally acknowledged. This is because anyone looking at the evidence can see that the sceptics are often totally unreasonable - whether they are ultimately right or not! There is a famous case where one piece of telepathy research was criticised on the grounds that the subjects could have climbed through an air conditioning vent in order to communicate!

Clearly if PSI exists, it must almost certainly be dependant on mood and ambience, and additional experimental rigour can obviously come at the expense of these intangibles. Critics such as Mszlazak never seem to take this into account.

If this were an ordinary part of science, there would be hundreds of research efforts working with animals - because Rupert Sheldrake seems to have discovered experiments with a particularly strong effect. The presentiment effect would also have been repeated (or not) in many laboratories, and any possible statistical flukes would have been dealt with. For example, as discussed earlier, there is a strange effect in which random variations in the number of events (e.g. disturbing images) can couple with an arousal model (where subjects start to anticipate the next event 'using' the gamblers fallacy). One obvious way to deal with that would be to use a randomisation technique that always displayed the same number of disturbing images - with no sample fluctuations. However, the sceptics don't have to agree on best practice - if that were done, the cry would undoubtedly go up that the data was no longer totally random.

In an ordinary area of science, I guess there are three groups:

1) Those that don't believe in the subject matter.

2) Those that are not sure, but interested.

3) Those that are pretty sure they are on to something.

The normal process is that people in group 1 get on with something else. Those in groups 2 and 3 do the work and criticise each other's work as they see fit. If the effect is real, the growing evidence gradually converts people from group 1, if the results don't add up, more and more people drift into group 1. I wish this process could happen with PSI.

David

[Topher Cooper]

Hey Topher, forget the history lesson!

I'd rather not, but I can see why you would want me to. The history contradicts your claims. What you say would be convincing -- a single, large, well controlled study with strong positive results -- has already been done, so you would rather not allow history.

The state-of-the-art thinking about meta-analysis is it CANNOT be used for corroberation or confirmation. They are only exploratory in the best circumstances.

I'm not so negative about science as you. Meta-analysis is the name given to any method of evaluating multiple sources of information. What you are saying boils down to that there must be one initial, large, perfectly controlled study. After that, there can be no further experiments. Corroboration or confirmation would be irrelevant since that would be meta-analysis and they cannot either strengthen or weaken the initial experiment without being part of a meta-analysis (whether formal or otherwise).

Writing from my hotel lobby on Block Island, RI I can't check your reference, but I suspect that you are making the same error that Ray Hyman, in arguing against putting his general claims about parapsychology to any sort of test.

Basically, there are two types of meta-analysis, which must be distinguished. The first type combines multiple experiments, each of which used comparable methods and were designed to test the issue that the meta-analysis is designed to test. This is the kind of analysis that we are speaking about, and it would be shocking indeed if it were not valid. It would mean, as I said that evidence from more than one experiment could never be taken in any formal sense as supporting each other.

The other type of experiment involves taking experiments that were not designed to test a particular hypothesis now under consideration and treating them as representing different conditions in a new test. An example of this would be if we take a bunch of telepathy experiments each of which happened to have a different distance between the agent and the percipient and attempt to reach some conclusion about the effects of distance on telepathy. It is hardly the state of the art but an obvious fact that conclusions from this kind of meta-analysis can only be suggestive (i.e., the analysis will be exploratory) rather than conclusive. This is so because we cannot fully control that all other factors are uniform between conditions when no care has been taken to assure this. For example, telepathy experiments at greater distances may tend to use batch rather than individual feedback, or a greater time before feedback occurs, or a less engaging feedback.

This later type can, indeed, only be used to suggest more rigorous experiments or to support such experiments that have already taken place. Despite the obviousness of this, articles sometimes appear with more enthusiasm than is justified and it needs to be brought up regularly. This is a far cry, though, from your claim that one can never analyze more than one experiment bearing on the same hypothesis. That would not be an obscure result but something that would be a major revolution in, for example, evidence based medicine or refining estimates of the properties of elementary particles.

[mszlazak]

Topher, you're now just playing little games or you don't want to face the harsh reality. Also, your notion that meta-analysis is one prospective experiment is silly.

Here are the two articles I mentioned:

A PROPOSAL AND CHALLENGE FOR PROPONENTS AND SKEPTICS OF PSI

Letter on meta-analysis in parapsychology

Cheers.