Announcement

Collapse
No announcement yet.

Ganzfeld Effect Size

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Ganzfeld Effect Size

    I saw this article by Ray Hyman the other day where he reiterates his views that there is no psi effect, psi hasn't been demonstrated by parapsychologists, and effect sizes are declining to zero.

    Anomalous Cognition? A Second Perspective by Ray Hyman (2008)
    http://www.csicop.org/si/2008-04/hyman.html

    It prompted me to track down this meta-analysis :

    "Updating the Ganzfeld Database" by Daryl J. Bem, John Palmer, and Richard S. Broughton (2001)

    pdf
    http://dbem.ws/Updating_Ganzfeld.pdf
    google html
    http://74.125.95.104/search?q=cache:...lnk&cd=2&gl=us

    In updating "Updating the Ganzfeld Database" the authors describe a meta-analysis restricted to studies that all used the same "standard" protocol, they found a real effect that is not explainable by chance. This study seems to refute the "decline of effect size" criticism for ganzfeld experiments. They included 10 new studies following the standard protocol and the effect size went up. The effect size for those new studies in fact was relatively high - 36.7%.

    So I am wondering if anyone knows why this is not accepted as evidence for psi by the skeptical community? Why is there still a debate?

    I've taken a glance at
    A History of Psi in the Ganzfeld by Andrew Endersby (2005)
    http://www.skepticreport.com/psychicpowers/ganzfeld.htm

    Endersby admits he is not an authority on statistics but after doing his own analyses he is not very enthusiastic about there being a real ganzfeld effect.

    However, unless his analyses also were done with studies using the same "standard" protocol, I don't see how it can invalidate the conclusion of "Updating the Ganzfeld" that there is a real effect not explainable by chance found when the standard protocol is followed.

    So I am wondering if anyone might know if there is a study (preferably on line) that really refutes what "Updating the Ganzfeld" concludes about the standard protocol?

    If not, why do skeptics not accept that result?

    (There's a member of the forum with the name Ersby. Is that Endersby? If so and if you are reading this, I would be interested in your opinion on my question, and I apologise for referring to you in the third person.)
    Last edited by anonymous; November 16th, 2008, 02:01 PM.

  • #2
    Originally posted by anonymous View Post
    I saw this article by Ray Hyman the other day where he reiterates his views that there is no psi effect, psi hasn't been demonstrated by parapsychologists, and effect sizes are declining to zero.

    Anomalous Cognition? A Second Perspective by Ray Hyman (2008)
    Anomalous Cognition? A Second Perspective
    With regard to decline effects .... if psi is a more of a collective effect, than individual effect .... then the more people who cast doubt about the results, the more effect could decline? With something like 'telepathy' there is no solid logical reason to assume it must work like a private telephone call from individual to individual.

    Therefore psi experiments will probably do better when novel and new .... until boredom steps in and there is a skeptical backlash increasing doubt amongst experimenters and moving closer to the expectations of skeptics. Most parapsychologists are trained to be skeptical, they need to be.

    This is one of the reasons I want to see an esteemed placebo vs non-esteemed placebo type of trial. It wouldn't surprise me if the effects follow the experimenters expectations (not just the patients expectations) even under a double blind protocol. ... until they find out the esteemed drug was in fact just a placebo, then the effect will gradually decline towards zero.

    So I am wondering if anyone might know if there is a study (preferably on line) that really refutes what "Updating the Ganzfeld" concludes about the standard protocol?

    If not, why do skeptics not accept that result?
    I think they want a stronger effect (and will use the 'extraordinary evidence' argument) ... which is why I spent time arguing why brain filters mind theory actually predicts a weak psi effect and only a erratic stronger effect.
    Last edited by Open Mind; October 18th, 2009, 11:14 AM.

    Comment


    • #3
      Originally posted by Open Mind View Post
      With regard to decline effects .... if psi is a more of a collective effect, than individual effect .... then the more people who cast doubt about the results, the more effect could decline? With something like 'telepathy' there is no solid logical reason to assume it must work like a private telephone call from individual to individual.

      ...

      I think they want a stronger effect (and will use the 'extraordinary evidence' argument) ... which is why I spent time arguing why brain filters mind theory actually predicts a weak psi effect and only a erratic stronger effect.
      When you say "collective" I'm not sure what you mean. Do you think the strength of an individual's psi is dependent on collective factors? Or, that an individual's psi interacts another person's psi in ways we are not aware of, for example if people are all just a tiny bit telepathic then the conventional wisdom in a group of people may effect people's beliefs subliminally. Or, did you have something else in mind?

      Do you think that psi as a collective effect is related to the brain as a filter model, or are they just two independent aspects? How does natural selection of the "filter" fit in with your views?

      My working hypothesis is that psi abilities are how spirits normally interact with each other and their environments but those abilities don't work well when limited by the body.

      It's like if I put on a heavy suit of armor my vision is cut down by the slit in the visor and my running speed is slowed down by the weight. If it gets dented in the elbow joint then I might lose the ability to flex my elbow. But if I get out of the armor all my normal abilities return.

      Variations in psi ability could be due to variations in spirits and bodies. Some people might not be strong enough to run in the armor while others might be able to albeit slowly. Some suits of armor might be lighter than others.

      I would say psi seems to run in familes because some suits of armor are lighter some have wider visor slits, etc. Capabilities that the spirit doesn't already have can't occur due to variations in the body so evolution is limited. It may also be that psi doesn't evolve in greater strength because life on earth has already evolved to allow the maximum leakage of psi that is possible. There might also be negative selective pressures such as insanity.

      Comment


      • #4
        I think the ganzfeld debate (and related meta-analyses) is a nice example of individual researchers seeing what they want to see in the data. The extent to which a priori worldviews determine the outcome of even 'objective' data analyses is frequently grossly underestimated. Any analysis involves a dozen or more 'judgment calls' (data inclusion or exclusion, judging criteria, statistical technique, etc.) that will bias the outcome.

        In the first issue of the Zetetic Scholar, Laurent Beauregard tried to quantify this a priori bias in a Bayesian model. This model deserves far more attention that it currently receives. This issue available here. (Issue no 1., 1978, p. 3).

        In short, finding a consistent level of proof that's good enough for both advocates and skeptics is probably futile because both parties have such (sometimes wildly) differing worldviews, and because the phenomena under discussion is subtle and ambiguous enough to be explained in both 'skeptical' or 'advocate's' terms.

        For example, I personally think that the ganzfeld evidence points fairly well to their being some genuine anomalous effect, but then my own personal philosophy lacks the sense of philosophical outrage that many skeptics display towards 'psi.' I personally can see no strong reasons why minds shouldn't be able to communicate with each other by unknown means occasionally. I also think that the versions of 'science' supported by some skeptics seem rather old-fashioned, especially in the light of non-locality, etc. So I'm philosophically speaking biased in a favorable direction.

        My point is that in a sense one has already made a significant mental 'decision' about a phenomena BEFORE you even look at the data. The outcome of experiments will, of course, affect this, but only over a longer run and also depending on lots of contextual factors that include the company you keep, what you read or hear regularly etc. In other words,world views, most certainly INCLUDING rational-scientific ones are at least partly the outcome of brainwashing!

        Comment


        • #5
          Originally posted by Matt Colborn View Post

          In the first issue of the Zetetic Scholar, Laurent Beauregard tried to quantify this a priori bias in a Bayesian model. This model deserves far more attention that it currently receives. This issue available here. (Issue no 1., 1978, p. 3).
          Hi Matt,

          Intersting article.

          Please correct me if I'm wrong, but is seems to me that Beauregard is expressing Hume's philosophy that "extraordinary claims require extraordinary proof" in a mathematical equation of probabilities. He agrees the assignment of values for probabilities is subjective. He shows that the skeptic's assignment of values will tend to make it hard to confirm psi and the believers assignments will make it hard to disprove psi and states this demonstrates the equation is truly a mathematical expression of Hume's philosophy.

          What I don't get is what is the use of this? If the assignments of values are subjective how is it useful?

          He concludes that scientific objcetivity requires a skeptics bias. I don't follow the logic that explains how he reaches that conclusion.

          What am I missing?

          Thanks

          Comment


          • #6
            Where is the Ganzfeld Database?

            Originally posted by anonymous View Post
            I saw this article by Ray Hyman the other day where he reiterates his views that there is no psi effect, psi hasn't been demonstrated by parapsychologists, and effect sizes are declining to zero.

            Anomalous Cognition? A Second Perspective by Ray Hyman (2008)
            Anomalous Cognition? A Second Perspective

            It prompted me to track down this meta-analysis :

            "Updating the Ganzfeld Database" by Daryl J. Bem, John Palmer, and Richard S. Broughton (2001)
            Can someone tell me, is the Ganzfeld Database an actual database, or just more post-hock meta-analysis? If it's real, who keeps it? Have they put it on line? What's the protocol for ensuring the integrity of updates?

            Originally posted by anonymous View Post
            In updating "Updating the Ganzfeld Database" the authors describe a meta-analysis restricted to studies that all used the same "standard" protocol, they found a real effect that is not explainable by chance.
            Of course it's not all chance; it includes Dalton (1997) which got p=.00000007. Has anyone of any repute seriously suggested that one was just chance?

            Originally posted by anonymous View Post
            This study seems to refute the "decline of effect size" criticism for ganzfeld experiments. They included 10 new studies following the standard protocol and the effect size went up. The effect size for those new studies in fact was relatively high - 36.7%.
            No it was not. If you do not know the difference between hit rate and effect size, then you are wasting your time reading these papers.

            Originally posted by anonymous View Post
            So I am wondering if anyone knows why this is not accepted as evidence for psi by the skeptical community? Why is there still a debate?
            No science would accept such contradictory results. Other fields use meta-analysis while better data is unavailable, but they understand it is unreliable.

            When is parapsychology going to start acting like a science? Will they ever state definite, predicative theories that could be refuted by experiment? J. B. Rhine published Extra-Sensory Perception After Sixty Years back in 1940, yet parapsychology still lacks any repeatable demonstration that the subject they are studying even exists.

            -Bryan

            Comment


            • #7
              Originally posted by BannedBySkeptiko View Post
              When is parapsychology going to start acting like a science?
              Bryan , why not look at parapsychology as the test of the 'scientific method' ... without parapsychology, the scientific method enters unfalsifiability, therefore becomes unscientific ... take part in it, enjoy it

              I might comment later on your other points above ... need to go out.

              Comment


              • #8
                Originally posted by anonymous View Post
                What I don't get is what is the use of this? If the assignments of values are subjective how is it useful?
                I just thought of a way to make this equation useful.

                Factor in cognitive dissonance to correct for the subjectivity in probability assignments.

                Someone suffering from congnitive dissonance, (as measured by personal attacks, emphatic statements and over reacting) will make less accurate probability assessments of the value of the evidence in proportion to their level of congnitive dissonance.

                However, the probability assignments can be normalized by modifying them in proportion to the amount of cognitive dissonance exhibited.
                Last edited by anonymous; November 17th, 2008, 05:59 PM.

                Comment


                • #9
                  Originally posted by anonymous View Post
                  but is seems to me that Beauregard is expressing Hume's philosophy that "extraordinary claims require extraordinary proof" in a mathematical equation of probabilities. He agrees the assignment of values for probabilities is subjective. He shows that the skeptic's assignment of values will tend to make it hard to confirm psi and the believers assignments will make it hard to disprove psi and states this demonstrates the equation is truly a mathematical expression of Hume's philosophy.

                  What I don't get is what is the use of this? If the assignments of values are subjective how is it useful?

                  He concludes that scientific objcetivity requires a skeptics bias. I don't follow the logic that explains how he reaches that conclusion.

                  What am I missing?

                  Thanks
                  (1) You are quite correct, it is a formalization of Hume's arguments, although it's a level more sophisticated because in Hume's original argument the probability of miracles is fixed at extremely unlikely.
                  (2) The use is that it starts to take into account a priori probabilities, which traditional statistics do not. (This is why Bayesian methods are often seen as suspicious). However, Matthews (1999) makes a good case that significance levels assigned by traditional p values significantly exaggerate the probability that the alternative hypothesis can be rejected. Bayesian statistics actually produce more conservative significance levels.
                  (3) "He concludes that scientific objectivity requires a skeptics bias" Yeah, I don't get why he says this either. I presume that he self-defines as a skeptic, and that's what drives him to say this. I think he's wrong; if you assign a sufficiently high level of unlikelihood to a phenomena, then no amount of evidence is going to change your mind.

                  In short, Bueregard's model helps make the subjective (judgement) elements of the scientific process explicit.


                  Matt.

                  Reference
                  Matthews, R.A.J. (1999) Significance levels for the assessment of anomalous phenomena. Journal of Scientific Exploration 13. 1--7.

                  Comment


                  • #10
                    I doubt if Ray Hyman or any member of CSI would concede the evidence for psi - even the $20 million CIA experiment hasn't swayed him.

                    But one criticism of the ganzfield studies is that they were never published in peer-reviewed journals. Or have they?

                    Comment


                    • #11
                      Originally posted by Open Mind View Post
                      Bryan , why not look at parapsychology as the test of the 'scientific method' ... without parapsychology, the scientific method enters unfalsifiability, therefore becomes unscientific ... take part in it, enjoy it
                      How did your issues with the scientific method become my task to test?

                      --Bryan

                      Comment


                      • #12
                        Originally posted by BannedBySkeptiko View Post
                        Of course it's not all chance; it includes Dalton (1997) which got p=.00000007. Has anyone of any repute seriously suggested that one was just chance?
                        Of course not.

                        Yet, when this study is included in an analysis with 39 other studies, the overall p-value drops by roughly 5 orders of magnitude.

                        Do you think that inclusion of the Dalton study guarantees a significant meta-analysis?

                        There is the possibility that after another 40 studies the p-value may drop by a similar order of magnitude. Then again, maybe it won't.

                        Comment


                        • #13
                          Originally posted by davidsmith73 View Post
                          Of course not.

                          Yet, when this study is included in an analysis with 39 other studies, the overall p-value drops by roughly 5 orders of magnitude.
                          Which is a much weaker argument that the results are not all due to chance.

                          Originally posted by davidsmith73 View Post
                          Do you think that inclusion of the Dalton study guarantees a significant meta-analysis?

                          There is the possibility that after another 40 studies the p-value may drop by a similar order of magnitude. Then again, maybe it won't.
                          The combined p-value is a the odds of getting over-all results that extreme if every single study's results are due to chance alone. What question(s) do you think that figure can answer, given that we already have strong evidence that the results are not all due to chance?

                          -Bryan

                          Comment


                          • #14
                            Originally posted by BannedBySkeptiko View Post
                            What question(s) do you think that figure can answer, given that we already have strong evidence that the results are not all due to chance?

                            The Dalton study was 128 trials with results that were 14 million to one against chance. Yes, taken alone this is huge odds against chance.

                            You might remember that in a previous discussion, you implied that millions of trials are necessary to nullify the Dalton study. This is simply not true because the Dalton p-value is calculted in relation to the hit rate achieved within a specific number of trials. When you add more trials, that local deviation no longer applies.

                            When the 128 trials of the Dalton were added to only 1533 trials specified in the Bem, Palmer and Broughton paper, the odds against chance fell by roughly 5 orders of magnitude.

                            How many more trials would have been needed to nullify the Dalton study?

                            Certainly not millions.


                            So the meta-analysis can suggest whether the overall results are likely due to chance alone. It would be foolish not to conduct a meta-analysis and rely on the Dalton study, or a couple more, when there are so many non-significant and negative studies.


                            Should the Dalton study be treated as an outlier?

                            I don't think so. From a cursory glance at the appendix, it looks like the study has an effect size in line with the normal distribution of the whole.

                            Comment


                            • #15
                              Originally posted by davidsmith73 View Post
                              The Dalton study was 128 trials with results that were 14 million to one against chance. Yes, taken alone this is huge odds against chance.

                              You might remember that in a previous discussion, you implied that millions of trials are necessary to nullify the Dalton study.
                              I had suspected you had no clue what I was saying; now I've confirmation.

                              Originally posted by davidsmith73 View Post
                              This is simply not true because the Dalton p-value is calculted in relation to the hit rate achieved within a specific number of trials. When you add more trials, that local deviation no longer applies.

                              When the 128 trials of the Dalton were added to only 1533 trials specified in the Bem, Palmer and Broughton paper, the odds against chance fell by roughly 5 orders of magnitude.

                              How many more trials would have been needed to nullify the Dalton study?

                              Certainly not millions.
                              Can you cite a reference on what nullifying a study means?

                              Suppose the first study of some phenomenon had gotten z=6.2. Now if the second study reports z=-6.2, does that mean the first becomes null?

                              Originally posted by davidsmith73 View Post
                              So the meta-analysis can suggest whether the overall results are likely due to chance alone.
                              The problem is that some are treating it as indicating consistency, which it does not.

                              I don't know anyone who still argues that the entire collection of ganzfeld outcomes is due to chance alone.


                              Originally posted by davidsmith73 View Post
                              It would be foolish not to conduct a meta-analysis and rely on the Dalton study, or a couple more, when there are so many non-significant and negative studies.


                              Should the Dalton study be treated as an outlier?

                              I don't think so. From a cursory glance at the appendix, it looks like the study has an effect size in line with the normal distribution of the whole.
                              The hope had been, and some parapsychologists may still be at this, to use the mix of outcomes to discover the important variables to control in order to formulate a repeatable demonstration. The effect sizes found in ganzfeld studies look nothing like what we would expect from studying a real phenomenon.

                              I had not seen the appendix with this normal distribution analysis. Is it available on-line, and can you point me to it?

                              -Bryan

                              Comment

                              Working...
                              X