Forum Skeptics Explain Why They Believe Psi Doesn't Exist ...

[davidsmith73]

Originally Posted by Miguel

2 questions in advance:
You seem to believe that Sheldrake's work suggests telepathy. Is that right?

No. I'm just saying that the Sheldrake method is better. There is more chance of detecting an effect.

Yes!

Glad you agree with me.

If you refer back to the 2nd paper you will find that the figures also contain time waiting where visits caused by distraction were removed. I do not understand what comparison you mean.

You were claiming that the 12 trial experiment showed inexplicably more early 'signals' than the preliminary study. Even when the distractions are taken out of the 12 trial data, the graph shows about 20% of 'clean' waiting was done during the pre-return period. This is comparable to the preliminary experiment, if you wish to make such a comparison. So you really have no grounds for your claim here.

You are saying that the discrepancy is explained by the difference in how signalling is recognized. Am I understanding you right?

You are claiming there is a discrepancy. However, there is no reason to think that this is the case. The problem has arisen because you are comparing results between two experiments that used different methods and you are not taking that into account. In addition, the actual data contradicts what you are saying.

This is not a question of experimental design! Whether or not the dog signals is a matter if interpreting the dog's behavior!
Remember that in the preliminary paper Sheldrake relies on the parent's judgement!

Of course it is a question of experimental design. If W&S had consulted RS about their criterion, perhaps RS would have pointed out its flaws.

And what happens when the dog creates a false alarm before the time? W&M judge the trial to be a failure. What does Sheldrake do?

The 'Sheldrake method' records the total amount of time spent in the waiting area (refinements of this kind of analysis can be done of course, eg, dividing up this time into bins and doing a regression or something like that). This data can then be compared to control trials. This kind of method doesn't deem a trial to be a failure simply because the dog is a little anxious early on in the trial.

I would have thought that the primary way to deal with noise is to increase the sample size.

Yes you can do that too. To be fair, W&M's method would probably work in the long run but it would need more trials than the Sheldrake method to show an effect for the reasons we've discussed. However, 4 trials was too low to be statistically reliable.

Sorry but you are simply mistaken. I googled this up for you: http://www.cimt.plymouth.ac.uk/proje...stats_ch10.pdf
I hope it helps.

Sorry, which part that pdf do you think supports what you're saying about the lightbulbs and which bit shows that I'm wrong?

[Miguel]

Originally Posted by davidsmith73

No. I'm just saying that the Sheldrake method is better. There is more chance of detecting an effect.

Alright. You don't think Sheldrake's data suggests telepathy. Despite Sheldrake saying that.
You also say that Sheldrake's method is better at detecting an effect.

So is what you are saying basically that Sheldrake's method will detect an effect where there is none? (IE that it will routinely produce type I errors)

Glad you agree with me.

When you say what I am saying then I have hardly a choice.

You were claiming that the 12 trial experiment showed inexplicably more early 'signals' than the preliminary study. Even when the distractions are taken out of the 12 trial data, the graph shows about 20% of 'clean' waiting was done during the pre-return period. This is comparable to the preliminary experiment, if you wish to make such a comparison. So you really have no grounds for your claim here.
You are claiming there is a discrepancy. However, there is no reason to think that this is the case. The problem has arisen because you are comparing results between two experiments that used different methods and you are not taking that into account. In addition, the actual data contradicts what you are saying.

I have explained how we can see that the later randomized experiments show more early signalling. If you disagree then reread my previous posts and show how my argument is wrong.

Of course it is a question of experimental design. If W&S had consulted RS about their criterion, perhaps RS would have pointed out its flaws.

Do I understand correctly: You are saying that Sheldrake is better at recongizing Jaytee's signalling behavior than his owners.

The 'Sheldrake method' records the total amount of time spent in the waiting area (refinements of this kind of analysis can be done of course, eg, dividing up this time into bins and doing a regression or something like that). This data can then be compared to control trials. This kind of method doesn't deem a trial to be a failure simply because the dog is a little anxious early on in the trial.

Are you saying that you disagree with what Sheldrake did?

Yes you can do that too. To be fair, W&M's method would probably work in the long run but it would need more trials than the Sheldrake method to show an effect for the reasons we've discussed. However, 4 trials was too low to be statistically reliable.
Sorry, which part that pdf do you think supports what you're saying about the lightbulbs and which bit shows that I'm wrong?

Understanding 10.1 and maybe 10.2 should be sufficient.
Come to think of it. If you have some favorite math book that covers statistics look up binomial test.

[davidsmith73]

Originally Posted by Miguel

Alright. You don't think Sheldrake's data suggests telepathy. Despite Sheldrake saying that.

I am just saying that Sheldrakes methods are better than W&M's. There are some specific problems with Sheldrakes design as he reported it such as the non-random interspersion of control and experimental trials and for that reason I would be hesitant on concluding telepathy from this study. But the general method of using the continuous dependent variable of waiting time over the trial period is superior to W&S's method for all the reasons we've discussed.

You also say that Sheldrake's method is better at detecting an effect.

So is what you are saying basically that Sheldrake's method will detect an effect where there is none? (IE that it will routinely produce type I errors)

Sheldrakes methods will detect an effect and could be regarded as telepathy if adequate controls are in place, like I said before. This is in contrast to W&M's methods which are likely to increase noise.

I have explained how we can see that the later randomized experiments show more early signalling. If you disagree then reread my previous posts and show how my argument is wrong.

Firstly, the 12 trial experiment you are refering to shows 18% of waiting time during the pre-return period in the 'clean' trials. If the preliminary experiment shows 17% then the data contradict what you are saying straight off. Secondly, the methods differ between the two experiments making comparison difficult. In summary, there are no grounds for your claim that the later randomized experiments show more early signalling.

Look, I know you are desperate to defend the W&M experiment. To be honest, it has its plus points. But unfortunately, it had a few major problems - the 2 second success/failure criterion and only 4 trials.

Do I understand correctly: You are saying that Sheldrake is better at recongizing Jaytee's signalling behavior than his owners.

I'm saying he is in a better position to recognise the implications of W&M's criterion of trial success/failure for the experiment.

Understanding 10.1 and maybe 10.2 should be sufficient.

I've looked through 10.1 and 10.2 and don't see anything that helps your light bulb argument or shows where I am mistaken.

[Miguel]

Originally Posted by davidsmith73

I am just saying that Sheldrakes methods are better than W&M's. There are some specific problems with Sheldrakes design as he reported it such as the non-random interspersion of control and experimental trials and for that reason I would be hesitant on concluding telepathy from this study. But the general method of using the continuous dependent variable of waiting time over the trial period is superior to W&S's method for all the reasons we've discussed.

I disagree on this. I think it is better using a method that allows one to conclude the existence of a phenomenon than one that does not.
However, I do agree that your proposal would be superior to W&M's approach.
Sheldrake's finding that Jaytee does not spend an equal time waiting at all times is eventually meaningless and does not allow any conclusion.

Firstly, the 12 trial experiment you are refering to shows 18% of waiting time during the pre-return period in the 'clean' trials. If the preliminary experiment shows 17% then the data contradict what you are saying straight off. Secondly, the methods differ between the two experiments making comparison difficult. In summary, there are no grounds for your claim that the later randomized experiments show more early signalling.

I had some time to chill and to reappraise my argument. While the later experiments do show more early signalling there is no basis for claiming that this is significant.
I simply decided to take the 17% result from Sheldrake's early paper as an assertion that there should never be more than that. This was neither fair nor reasonable and probably emotionally motivated. I therefore retract there claim that there is a discrepancy requiring explanation.

Look, I know you are desperate to defend the W&M experiment. To be honest, it has its plus points. But unfortunately, it had a few major problems - the 2 second success/failure criterion and only 4 trials.

There was no 2 minute success/failure criterion.

[David Bailey]

Originally Posted by Miguel

I disagree on this. I think it is better using a method that allows one to conclude the existence of a phenomenon than one that does not.
However, I do agree that your proposal would be superior to W&M's approach.
Sheldrake's finding that Jaytee does not spend an equal time waiting at all times is eventually meaningless and does not allow any conclusion.

You have to step back a bit and imagine you were designing a non-psi experiment involving dogs. Suppose you had owners blow a dog whistle to indicate their return, and you wanted to know if the dog had heard the signal. Perhaps the idea would be to get the owners to blow the whistle further and further away from the house to determine the limits of their hearing.

Now, you want the experiment to be as uncontaminated by noise as possible - you certainly don't want the odd cat to spoil the results! You would probably start by observing how the dog behaved, and if you saw him visit the window occasionally, and then intensively after the whistle had been blown, you would design the test accordingly.

The obvious way to proceed would be to let someone else judge blind exactly when the dog began intensive waiting at the window.

The Wiseman approach would mess up this conventional experiment, just as much as it messed up Rupert's experiment! Reducing experimental noise is an important part of many scientific experiments.

David

[Miguel]

Originally Posted by David Bailey

You have to step back a bit and imagine you were designing a non-psi experiment involving dogs. Suppose you had owners blow a dog whistle to indicate their return, and you wanted to know if the dog had heard the signal. Perhaps the idea would be to get the owners to blow the whistle further and further away from the house to determine the limits of their hearing.

Now, you want the experiment to be as uncontaminated by noise as possible - you certainly don't want the odd cat to spoil the results! You would probably start by observing how the dog behaved, and if you saw him visit the window occasionally, and then intensively after the whistle had been blown, you would design the test accordingly.

The obvious way to proceed would be to let someone else judge blind exactly when the dog began intensive waiting at the window.

The Wiseman approach would mess up this conventional experiment, just as much as it messed up Rupert's experiment! Reducing experimental noise is an important part of many scientific experiments.

David

What I would do:
If the dog never visited the window anyway I'd think the control group unnecessary. If the dog comes to the window immediately following the whistle I'd conclude he hears.

If the dog did visit the window occasionally we'd need that control. Sometimes, randomly, we'd just pretend to whistle and observe the dog's behavior then.
We'd compare the frequency of visits in both conditions.
(Of course, there's a number of finer points I omited for brevity like randomizing the time of the whistle and preplanning the number of trials)
I guess this 2nd possibility is what you have in mind.
I agree that this would be the best possible way.

However, this requires running a larger number of trials.

So what to do if you're strapped for time or money and want to get by with as few trials as possible? Well, you also happen to notice that the dog only makes rather short visits but when he hears the whistle he's supposed to go and stay at the window for a while. So you just time if the dog stays there for long enough and presto, the odd cat is out. It's quick, it's dirty, it's not the best solution but it works.
Basically by ignoring all the shorter visits it's as if the dog never visited on it's own.

[David Bailey]

Originally Posted by Miguel

So what to do if you're strapped for time or money and want to get by with as few trials as possible? Well, you also happen to notice that the dog only makes rather short visits but when he hears the whistle he's supposed to go and stay at the window for a while. So you just time if the dog stays there for long enough and presto, the odd cat is out. It's quick, it's dirty, it's not the best solution but it works.
Basically by ignoring all the shorter visits it's as if the dog never visited on it's own.

Exactly - and that is not what Wiseman did - he took the time of the first visit to the window as the only data! I am sure Rupert's data could be analysed the way you suggest - my point is, that in any experiment you absolutely have to take into account possible sources of noise, and that is what Wiseman didn't do - basically because the null result was fine by him!

David

[davidsmith73]

Originally Posted by Miguel

If the dog did visit the window occasionally we'd need that control. Sometimes, randomly, we'd just pretend to whistle and observe the dog's behavior then.
We'd compare the frequency of visits in both conditions.
(Of course, there's a number of finer points I omited for brevity like randomizing the time of the whistle and preplanning the number of trials)
I guess this 2nd possibility is what you have in mind.
I agree that this would be the best possible way.

However, this requires running a larger number of trials.

So what to do if you're strapped for time or money and want to get by with as few trials as possible? Well, you also happen to notice that the dog only makes rather short visits but when he hears the whistle he's supposed to go and stay at the window for a while. So you just time if the dog stays there for long enough and presto, the odd cat is out. It's quick, it's dirty, it's not the best solution but it works.
Basically by ignoring all the shorter visits it's as if the dog never visited on it's own.

I think the first method is more appropriate because it can better account for variability in the data.

For the first method, I think an even better way would be to look at the difference in the freq/duration of visits during a 'pre-whistle' and 'whistle' period for both conditions. If there is a real effect, this difference should be significantly higher in the experimental group. Looking at this difference, rather than the total frequency, can take noise caused by general anxiety into account, if we assume that this noise is equally distributed in our two conditions.

There's also no reason to assume that the latter method would need fewer trials to detect an effect. It depends on the variability present in the data and criterion you set for determining whether a trial is a success or failure. Set this criterion too strictly and you run the risk of detecting noise rather than your effect. For example, lets say that early on in the trial the dog is a bit excited and for some reason, goes to wait for a while and then leaves. This would be regarded as a fail if the waiting time is more than the required criterion. However, in the first method, this early wait would have much less of an effect because (presumably) the effect would manifest properly for the remainder of the trial.

If one were pushed for time and money, I would use the former method, not the latter.

[Miguel]

Originally Posted by David Bailey

Exactly - and that is not what Wiseman did - he took the time of the first visit to the window as the only data! I am sure Rupert's data could be analysed the way you suggest - my point is, that in any experiment you absolutely have to take into account possible sources of noise, and that is what Wiseman didn't do - basically because the null result was fine by him!

David

That's not quite true. If you look at Wiseman's paper you will find that all the visits were recorded by a blinded observer.
Ways in which noise was addressed:
From the start: The visit had to be inexplicable / The blinded observer must not have found an obvious reason for the visit.
After the first experiment: The duration of the visit had to be at least 2 minutes.
After the second experiment: Further experiments are post-poned until winter.

In Sheldrake's commentary you will find the results of these experiments graphed in terms of seconds spent at the window per ten minute period.
The pattern shown in these graphs is that the dog spends a roughly constant proportion of time at the window until at some point the proportion gets higher and higher. The graph is reminiscent of an exponential curve.
One experiment that shows a completely different pattern is omitted.

In the first experiment the Wiseman signal happens during the flattish period. This supports the notion of a false alert.
However in the other 2 graphed experiments the Wiseman signal happens in at the beginning of the period during which the proportion increases.
It seems hard to put this down to noise.

[Miguel]

Originally Posted by davidsmith73

For the first method, I think an even better way would be to look at the difference in the freq/duration of visits during a 'pre-whistle' and 'whistle' period for both conditions. If there is a real effect, this difference should be significantly higher in the experimental group. Looking at this difference, rather than the total frequency, can take noise caused by general anxiety into account, if we assume that this noise is equally distributed in our two conditions.

Yes, makes sense regarding the telepathy hypothesis. WRT the whistle experiment it seems a bit convoluted. Either the dog comes or it doesn't, whether there is a increase in the frequency/duration in the general period after the whistle seems academic. Well, doesn't matter.

If one were pushed for time and money, I would use the former method, not the latter.

If you only have 4 trials it seems futile to split those into control/experimental. You're almost certainly not going to get a solid conclusion.
On the other hand if the dog manages to signal properly on at least 1 or 2 of the simple trials you at least have a reason to go on.