[Phronk]

Originally Posted by Rudism

I won't bother getting into another point/counter-point quote-fest... I'm not a mathematician, so I can't speak authoritatively on the methods they used.

Fair enough. But I hope you trust my mathematical knowledge enough to at least be undecided about the results (as I am) rather than believing there is some obvious and fatal flaw with them.

All I can say is that throwing 40% of the trials for no other reason than them not supporting your hypothesis is silly, and any amount of statistics you run the remaining results through to support your hypothesis is going to be overshadowed by that.

Well, I thought I made it clear that there was a good reason for throwing them out; they're not relevant to the question being asked.

It's also worth noting that, if you are right and there is no communication going on at all, and thus any "hits" are due to chance alone, then there would be absolutely no difference in the chances of finding significant results with or without the thrown out trials. If it was properly conducted and analyzed, the chance of incorrectly concluding there was an effect if there wasn't one is exactly 0.05, or 5%.

So, I repeat, throwing out the trials introduced NO BIAS. If there was no effect, it wouldn't help him falsely find evidence for one. This may be counterintuitive, but unless someone can see a fault in my reasoning, it's a fact.

Think of it this way. A guy claims he's really good at predicting coin flips, but only when it's sunny outside. He predicts coins in a room with a window and is videotaped. The coin flipper goes on flipping no matter what, but sometimes it gets cloudy outside. Later, the videotape is examined, and trials where it was cloudy out are thrown out, because that wasn't the guy's claim.

If he really was guessing and couldn't predict the future, would it help him to throw out certain trials? Nope. The baseline of 0.5 remains the same because the flipper is always random. It didn't bias him at all. And we don't need to know base rates of his guessing (he could guess heads every time for all we care) or base rates of cloud-vs-sunny (all it does is determine the sample size of valid trials) to know this.

Would it be convincing if he did consistently guess more than 50% of the time? Yes. If anything, the smaller sample size would make it harder to find an effect if it really did exist. Significantly above-chance guessing at any subset that is chosen independently from hit rate is evidence of predicting the future.

I also note that they threw out the word 'camera' from the list because the bird was saying it so often (due to cameras being used in the experiment). Wouldn't this effect have been cancelled out by the random permutation analysis which you claim does compare the hits against a baseline? Why throw it out if the bird's baseline speech patterns are already accounted for in the post-analysis? How do we know there aren't other words that the bird was using frequently enough to also warrant being thrown out (perhaps "flower," which accounted for almost half of the hits used in the final analysis)?

I agree, that is slightly fishy. Especially if it was done after the fact, when hit rates were already known (e.g., they noticed that trials with "camera" being said were almost always misses). But the excuse that there were cameras in the room seems reasonable, and "signals" directly in the room would indeed only interfere with the psychic "signal" from a distance if it existed, adding noise to the effect they're looking at.

Still, if this was replicated, I'd expect them to exclude camera from the start.