James Randi Challenges Skeptiko’s Telepathic Dog

[davidsmith73]

Originally Posted by stonesean

This is very exciting and I can't wait to see the results if the tests go forward, but just remember, your claim must be spelled out EXACTLY before the test begins. Not the dog "may" do this, or "may" do that.....the dog MUST do "x". I see an issue coming up with the criteria of what exactly constitutes a "hit" or a "miss".....

I can't see a problem with saying that the dog spends significantly more time in the waiting spot during the owners return compared to when the owner isn't returning. If the right controls are in place and the return times are randomised then that would consitute a "paranormal" effect.

[Topher Cooper]

Originally Posted by alextsakiris

We've talked about doing something like this, but we just don't have a firm enough handle on the phenomena at this point... need a lot more dogs... a lot more video.

You need to be real careful about this. Years ago Charles Tart pointed out that a well recognized and understood phenomena in psychology (the phrase "training to extinction" comes to mind, but I don't remember whether that was the accepted name for it or just a description) might be a contributor to the "decline effect" in parapsychology. Basically, if you reward and/or punish an organism on the basis of an essentially noisy "signal" then they will tend to learn some random set of characteristics, which can end up decreasing the number of actual hits. This essentially makes the results even more "noisy" which causes further degradation.

Charley did some experiments that demonstrated that this actually seemed to be occurring in psi experiments.

[Chris Noble]

Originally Posted by davidsmith73

I can't see a problem with saying that the dog spends significantly more time in the waiting spot during the owners return compared to when the owner isn't returning. If the right controls are in place and the return times are randomised then that would consitute a "paranormal" effect.

If the dog simply waits ten minutes and then starts "signalling" and remains there then it wil spend a higher proportion of time at the "waiting spot" when the owner is returning than when the owner isn't returning. This will happen with perfect controls and randomisation.

You can whine all you want about Wiseman's criteria being too strcit but you clearly need more rigorous criteria to demonstrate "psi" than those used by Sheldrake.

[Chris Noble]

Originally Posted by davidsmith73

Well, that's true. But scientists aren't in the business of testing someone elses claims.

If Sylvia Browne claims she is psychic then it is perfectly valid to test her specific claims. In the absence of a theory of "psi" what do you expect scientists to do? Test the infinite number of possible claims?

If you read Sheldrake's first paper he was indeed claiming that the time that Jaytee began signalling was correlated to the time that Pam began the return journey.

In 20 cases Jaytee reacted at the time PS set off, or within 2 minutes of this time (Table 1).

Figure 2 shows a linear regression implying that the dog's behaviour really does have predictive power.

They are supposed to actively investigate the world around us. And that's the position we should be adopting when discussing protocols for Alex's experiments and his potential challenge application. What would be the best way to find out if dogs really have a telepathic link with their owner? Wiseman et al's procedure just lets a load of noise into the experiment. If you simply count a 'miss' as the first instance that the dog goes to the waiting area during the non-return period, then you increase your chances of recording noise rather than signal. For example, an anxious moment or a distraction from a bird near the window.

I would find these post hoc excuses more convincing if they were applied more uniformly. For instance in Wiseman's first trial the dog only spend 15 seconds at the window where there was no obvious reason during the return period. Sheldrake didn't seem to mind if the dog was responding to cars pulling up outside the house and dogs walking by if it happened to be during the return period.

Surely there are better ways to measure this effect that are acceptable to both parties? The application of most statistical techniques in most experimental situations are going to have some drawbacks and problems. The point here is to devise the best method, not a perfect method.

What about comparing the time spent in the waiting area as a proportion of the total waiting time? Lets say the owner returns home after 100 min and takes 10 min to get home. Lets also say that the dog goes to the waiting area after 10min and stays there untill the owner returns. The proportion of time spent waiting during return as a proportion of total waiting time is 10% whereas the time spent waiting during non-return time as a proportion of total waiting time is 90%. Doing more trials would only lead to a significant effect in the opposite direction, which is what we would expect because in this example it's clear that the dog isn't displaying much psi! On the other hand, if the dog tends to wait only during the return period (ie, a real psi effect but not with the precision that certain sceptics seem to want!) then this proportional measure will detect the effect. Or could there still be an artifact here?

I'm not clear what you are saying here.

I think the main point is that it is imperative to have criteria for judging the success or failure of a trial set up before they are undertaken. It is imperative that these criteria really test the claims that are being made. It is inexcusable that Sheldrake has not done this in all this time.

If Alex actually applies for the Randi 1 million dollar challenge he will have to do this.

[Venom]

Originally Posted by Topher Cooper

You need to be real careful about this. Years ago Charles Tart pointed out that a well recognized and understood phenomena in psychology (the phrase "training to extinction" comes to mind, but I don't remember whether that was the accepted name for it or just a description) might be a contributor to the "decline effect" in parapsychology. Basically, if you reward and/or punish an organism on the basis of an essentially noisy "signal" then they will tend to learn some random set of characteristics, which can end up decreasing the number of actual hits. This essentially makes the results even more "noisy" which causes further degradation.

Charley did some experiments that demonstrated that this actually seemed to be occurring in psi experiments.

Sounds for me like saying: "be carefull, because you can have a few lucky trials, but after a while, of course, they will be a return toward the mean (or average)".

Well, that's kind of obvious. Alex should stop now that he's lucky, but with more and more trials and more and more dogs, things are going to look a lot less impressive. Well, even now, he's just showing on YouTube his best trials, the lucky ones, but doesn't show the rest. It's kinda a filedrawer effect. He's goal is to make believers trough YouTube, not to really inform people...

[Larry Boy]

Originally Posted by Venom

Well, even now, he's just showing on YouTube his best trials, the lucky ones, but doesn't show the rest. It's kinda a filedrawer effect. He's goal is to make believers trough YouTube, not to really inform people...

Where's the extraordinary evidence for those extraordinary claims?

[Larry Boy]

Originally Posted by Chris Noble

If the dog simply waits ten minutes and then starts "signalling" and remains there then it wil spend a higher proportion of time at the "waiting spot" when the owner is returning than when the owner isn't returning. This will happen with perfect controls and randomisation.

That's a good point. There surely must be a way to compensate for that, though? (Without using post hoc analyses, of course.)

How about this?

You separate the trial into ten minute intervals, then after the trial is complete, you take out the period in which the dog spends the most time at the window, and use the starting time of this period as the measure of success. If the start of the longest waiting period corresponds to the start of the return of the owner, the trial is a success.

It will look like this then:

Dog waiting
____________ _ _ _ _ _______________________
|10min|10min|10min|10min|10min|10min|10min|10min|

Owner returning
__________________________________________
|10min|10min|10min|10min|10min|10min|10min|10min|

Success!


Dog waiting
_________ _ _ _ _ _____________________________
|10min|10min|10min|10min|10min|10min|10min|10min|

Owner returning
_____________________________________________
|10min|10min|10min|10min|10min|10min|10min|10min|

Failure!


Of course you will have to randomize the return times to make sure the dog doesn't succeed just by starting waiting at the same time every trial.

[davidsmith73]

Originally Posted by Chris Noble

If Sylvia Browne claims she is psychic then it is perfectly valid to test her specific claims. In the absence of a theory of "psi" what do you expect scientists to do? Test the infinite number of possible claims?

I expect scientists to apply their skills and knowledge to find out what is really going on. In the case of Jaytee, I don't think Wiseman et al. did a very good job of this. Although I'll give them the benefit of the doubt and suggest they ran out of time and resources...

If you read Sheldrake's first paper he was indeed claiming that the time that Jaytee began signalling was correlated to the time that Pam began the return journey.

In 20 cases Jaytee reacted at the time PS set off, or within 2 minutes of this time (Table 1).

Figure 2 shows a linear regression implying that the dog's behaviour really does have predictive power.

But it's entirely possible, and probable, that any predictive power such as this would be masked in an analysis where trials are classed as a failure because the dog goes to the waiting area for a few moments during the non-return period. Sheldrake may well have been looking at the temporal precision of this effect, but he was wise not to adopt Wiseman et al's criteria of 'hit' and 'miss' because that just massively increases your chances of type II errors for reasons I outlined before.

I would find these post hoc excuses more convincing if they were applied more uniformly. For instance in Wiseman's first trial the dog only spend 15 seconds at the window where there was no obvious reason during the return period. Sheldrake didn't seem to mind if the dog was responding to cars pulling up outside the house and dogs walking by if it happened to be during the return period.

In the case of Sheldrake, I'm sure he was aware of these issues before he ran his experiments, so these "excuses" are not likely to be post-hoc. If you're talking about post-hoc in the sense of explaining why Wiseman et al's data didn't show an effect, I would call it "criticism" rather than "excuses".

The best way would be not to exclude any data due to distractions because you run into huge difficulties deciding what is a distraction and what is not. The best way would be to analyse the data in such away that noise causd by distractions are kept to a minimum.

I'm not clear what you are saying here.

I'm suggesting a way to analyse the data using the amount of time in the waiting area as the dependent variable. If you simply calculate the proportion of time spent in the waiting area during return and non-return periods then will get the artifact you were talking about. If we consider the dog behaviour in our 100 min trial example we've been using, this would result in 100% waiting time during the return period and 90% for the non-period. I am suggesting that the waiting time is calculated as a proportion of the total time in the waiting area across return and non-return periods. This would give 10% for the return period and 90% for the non-return period (ie., 100 min of waiting in the waiting spot, 90min during non-return but only 10min during the return period). This would guard against the artifact you were talking about, I think.

I think the main point is that it is imperative to have criteria for judging the success or failure of a trial set up before they are undertaken. It is imperative that these criteria really test the claims that are being made. It is inexcusable that Sheldrake has not done this in all this time.

No, it's not imperative at all. Not many behavioural experiments measuring a continuous variable would adopt a 'hit' and 'miss' approach. It's more sensible to compare means of the dependent variable between conditions.

If Alex actually applies for the Randi 1 million dollar challenge he will have to do this.

Now, that I may have to agree with you on. But I'm sure it will on JREF's insistence.

[davidsmith73]

Originally Posted by Larry Boy

That's a good point. There surely must be a way to compensate for that, though? (Without using post hoc analyses, of course.)

Isn't there a simpler way? Just calculate the waiting time during the return period as a proportion of total waiting time across the whole experimental period, and the same for the non-return time, then compare the two with some statistical analysis. If the effect is as strong as it seems in the videos then you could probably use non-parametric methods with less assumptions about the distrubution of the data.

[Open Mind]

Good to see Topher and David Smith back on this forum .... where have you guys been hiding? (Well David has been back a while)

Sean, good to you join forum too, the more diverse opinions the better.