[Rudism]

Originally Posted by Phronk

They were thrown out because the question being asked was "out of all the times when the bird said words from this preselected subset of his vocabulary, how many times did it line up with what his owner was seeing in another room?

This is equivalent to saying "the reason we threw out the results was because we wanted to see what happened with the results we didn't throw out." It doesn't answer the question why were they thrown out? I'll address this again later on.

Originally Posted by Phronk

If chance alone were operating, how likely would it be to see it match up this many times?" To answer that question, it's irrelevant to look at the trials where he wasn't saying words from that set.

This is true--but we also can't answer that question in general (I will explain this shortly). Additionally, trials where the bird doesn't say words from the set are relevant to the bigger question--namely, do the psychic messages exist at all?

Let's look at your question (what to expect by chance alone): without a baseline (which this study did not compute), it is impossible to answer this question. I'll try to illustrate with another example... Let's pretend the bird continuously talks throughout the entire experiment, just saying "ball bike flower" over and over again non-stop. If you show a random picture of either a ball, a bicycle, or a flower to the owner, then record what the bird says during the next two minutes, he's going to get a hit 100% of the time, even by chance. This should be proof enough that the bird's baseline speech patterns alone (regardless of any psychic phenomenon) can obviously affect the statistics. You need to take it into account before you can point to anything as a possible anomaly.

Originally Posted by Phronk

Let's simplify the study for illustration. Let's say there are only 3 pictures to choose from - a ball, a flower, and a bike. The bird has these three words in his vocabulary, but tends to say "ball" and "flower" way more than he says "bike."

We want to know if, when he says one of these 3 words, it tends to be the one his owner is looking at.

We do one hundred trials. The bird says one of the 3 words on 10 of those trials. We could look at all 100, but why? We're only interested in the trials where he said one of those 3 words.

Again, in my example, if all the bird says is "ball bike flower" every few seconds, he's going to get a hit on every single picture. You'll get a 100% hit rate by chance alone using that design. It's meaningless on its own.

Originally Posted by Phronk

To further illustrate why it is fair to throw out the other 90 trials, let's assume that only chance is operating. The bird is just babbling and is not influenced by the owner's thoughts at all.

On the first trial, the bird says "ball". What are the chances that he's right? Well, 1/3, because there's a 1/3 chance that the computer randomly selected the ball picture for the owner.

Wrong... The chance that you'll get a ball picture is 1/3. The chance that the bird will say "ball" cannot be calculated without knowing how often the bird says ball in general. Maybe he says ball every 1 minute on the minute, in which case, during a 2 minute period after showing the picture, there is a 100% chance that the bird will say "ball" during that trial. That means in your post analysis, regardless of what the other trials say, you are going to see a 100% hit rate on the ball pictures. Looks like psychic powers right? But it's obviously not in this case. You need the baseline to compare it to.

Originally Posted by Phronk

On the second trail, the bird says "ball" again. Still a 1/3 chance of being right.

He says "ball" yet again. Still only a 1/3 chance of being right.

He says "flower" for the next six trials. By chance, he should only be right on about 1/3 (i.e., 2) of them.

He finally says "bike". But the chances are STILL just 1/3 that he's right.

This would only be true if the bird is just guessing one word for each trial and gets it either right or wrong. That's not how the experiment was designed, though.

You are absolutely right when saying that we want to compare the actual results to what we would expect by chance alone. The only problem is that the experiment did not figure out what to expect by chance alone. I think I've sufficiently demonstrated this with my previous examples.

Originally Posted by Phronk

But in this case, the response to the stimulus - i.e., responding with matching words to the owner's "psychic" signal - is the evidence that the stimulus exists. Any response at all to the stimulus is evidence that it exists.

I absolutely agree with this statement. Along the exact same reasoning, trials during which the bird did not respond with words matching to the owner's "psychic" signal are evidence that the stimulus does not exist. But most of this data was thrown out, apparently using the very excuse that the bird did not appear to get any psychic messages during that trial! If that's not bias, I don't know what is.

Originally Posted by Phronk

Besides, if it skews results with parrots, it skews the results with children, and it's shoddy research in either case.

Even if an effect is well-established, studying its properties with biased research isn't going to reveal anything. If there are genuine problems with Sheldrake's research - and there very well may be - then researchers working with autistic kids and other animals should take note and avoid the same problems so that the effects they purport to discover aren't really nonexistent like the psi effects may be.

It all depends on the hypothesis being tested. If your hypothesis is that there are psychic messages being sent from owner to bird, then throwing out trials that are clearly evidence to the contrary is obviously biased. If, on the other hand, your hypothesis is that an autistic child will respond to stimulus X using method A more often than method B, then it is safe to throw out the trials where the child does not respond at all to stimulus X, because it doesn't say anything one way or another about the hypothesis (which is only interested in comparing responses).

See the difference? In the first case, every trial has important data that can answer the question being tested. We want to know, by looking at the bird's responses, if he's getting the stimulus at all. In the second case, only the trials where the child actually responds using either method A or method B are important to the question. If you were to do a study that hypothesizes autistic children will respond at all to stimulus X, then obviously throwing out trials where the child doesn't respond would be a ridiculous thing to do--just as it is in the parrot experiment.