Animal Communicator Research (Podcast)

[Rudism]

Originally Posted by Phronk

They were thrown out because the question being asked was "out of all the times when the bird said words from this preselected subset of his vocabulary, how many times did it line up with what his owner was seeing in another room?

This is equivalent to saying "the reason we threw out the results was because we wanted to see what happened with the results we didn't throw out." It doesn't answer the question why were they thrown out? I'll address this again later on.

Originally Posted by Phronk

If chance alone were operating, how likely would it be to see it match up this many times?" To answer that question, it's irrelevant to look at the trials where he wasn't saying words from that set.

This is true--but we also can't answer that question in general (I will explain this shortly). Additionally, trials where the bird doesn't say words from the set are relevant to the bigger question--namely, do the psychic messages exist at all?

Let's look at your question (what to expect by chance alone): without a baseline (which this study did not compute), it is impossible to answer this question. I'll try to illustrate with another example... Let's pretend the bird continuously talks throughout the entire experiment, just saying "ball bike flower" over and over again non-stop. If you show a random picture of either a ball, a bicycle, or a flower to the owner, then record what the bird says during the next two minutes, he's going to get a hit 100% of the time, even by chance. This should be proof enough that the bird's baseline speech patterns alone (regardless of any psychic phenomenon) can obviously affect the statistics. You need to take it into account before you can point to anything as a possible anomaly.

Originally Posted by Phronk

Let's simplify the study for illustration. Let's say there are only 3 pictures to choose from - a ball, a flower, and a bike. The bird has these three words in his vocabulary, but tends to say "ball" and "flower" way more than he says "bike."

We want to know if, when he says one of these 3 words, it tends to be the one his owner is looking at.

We do one hundred trials. The bird says one of the 3 words on 10 of those trials. We could look at all 100, but why? We're only interested in the trials where he said one of those 3 words.

Again, in my example, if all the bird says is "ball bike flower" every few seconds, he's going to get a hit on every single picture. You'll get a 100% hit rate by chance alone using that design. It's meaningless on its own.

Originally Posted by Phronk

To further illustrate why it is fair to throw out the other 90 trials, let's assume that only chance is operating. The bird is just babbling and is not influenced by the owner's thoughts at all.

On the first trial, the bird says "ball". What are the chances that he's right? Well, 1/3, because there's a 1/3 chance that the computer randomly selected the ball picture for the owner.

Wrong... The chance that you'll get a ball picture is 1/3. The chance that the bird will say "ball" cannot be calculated without knowing how often the bird says ball in general. Maybe he says ball every 1 minute on the minute, in which case, during a 2 minute period after showing the picture, there is a 100% chance that the bird will say "ball" during that trial. That means in your post analysis, regardless of what the other trials say, you are going to see a 100% hit rate on the ball pictures. Looks like psychic powers right? But it's obviously not in this case. You need the baseline to compare it to.

Originally Posted by Phronk

On the second trail, the bird says "ball" again. Still a 1/3 chance of being right.

He says "ball" yet again. Still only a 1/3 chance of being right.

He says "flower" for the next six trials. By chance, he should only be right on about 1/3 (i.e., 2) of them.

He finally says "bike". But the chances are STILL just 1/3 that he's right.

This would only be true if the bird is just guessing one word for each trial and gets it either right or wrong. That's not how the experiment was designed, though.

You are absolutely right when saying that we want to compare the actual results to what we would expect by chance alone. The only problem is that the experiment did not figure out what to expect by chance alone. I think I've sufficiently demonstrated this with my previous examples.

Originally Posted by Phronk

But in this case, the response to the stimulus - i.e., responding with matching words to the owner's "psychic" signal - is the evidence that the stimulus exists. Any response at all to the stimulus is evidence that it exists.

I absolutely agree with this statement. Along the exact same reasoning, trials during which the bird did not respond with words matching to the owner's "psychic" signal are evidence that the stimulus does not exist. But most of this data was thrown out, apparently using the very excuse that the bird did not appear to get any psychic messages during that trial! If that's not bias, I don't know what is.

Originally Posted by Phronk

Besides, if it skews results with parrots, it skews the results with children, and it's shoddy research in either case.

Even if an effect is well-established, studying its properties with biased research isn't going to reveal anything. If there are genuine problems with Sheldrake's research - and there very well may be - then researchers working with autistic kids and other animals should take note and avoid the same problems so that the effects they purport to discover aren't really nonexistent like the psi effects may be.

It all depends on the hypothesis being tested. If your hypothesis is that there are psychic messages being sent from owner to bird, then throwing out trials that are clearly evidence to the contrary is obviously biased. If, on the other hand, your hypothesis is that an autistic child will respond to stimulus X using method A more often than method B, then it is safe to throw out the trials where the child does not respond at all to stimulus X, because it doesn't say anything one way or another about the hypothesis (which is only interested in comparing responses).

See the difference? In the first case, every trial has important data that can answer the question being tested. We want to know, by looking at the bird's responses, if he's getting the stimulus at all. In the second case, only the trials where the child actually responds using either method A or method B are important to the question. If you were to do a study that hypothesizes autistic children will respond at all to stimulus X, then obviously throwing out trials where the child doesn't respond would be a ridiculous thing to do--just as it is in the parrot experiment.

[Phronk]

Originally Posted by Rudism

This is equivalent to saying "the reason we threw out the results was because we wanted to see what happened with the results we didn't throw out." It doesn't answer the question why were they thrown out? I'll address this again later on.

Well, something worth mentioning before I go on is that even if you're right and they shouldn't have been thrown out (though I still disagree), the results are still significant if they are analyzed your way. So arguing this point doesn't change the conclusions.

Let's look at your question (what to expect by chance alone): without a baseline (which this study did not compute),

Again, chance is the baseline. If we had some empirical baseline of how many times the bird says certain words on a regular basis, that would complicate the whole experiment and introduce more potential biases. Maybe the bird says "flower" a lot at home, but when in a lab and under stress, tends to say "flower" a lot less. Thus, comparing to the old flower baseline no longer applies and we'd find a statistical difference that has nothing at all to do with communication between the parrot and his owner. In a properly conducted study, testing deviations from chance is a much cleaner test of the hypothesis with less potential for bias.

---

Note that for the following examples, you are assuming that multiple responses in a trial were kept, so that the bird could have more than one "guess" per trial. I did not make this assumption in my reasoning to keep it simple, so it doesn't really apply. However, I looked at the original study (http://www.scientificexploration.org...ke_morgana.pdf) and they did indeed allow more than one guess per trial. While this complicates things, it does not change the general point of my previous post, as I will demonstrate below.

I'll try to illustrate with another example... Let's pretend the bird continuously talks throughout the entire experiment, just saying "ball bike flower" over and over again non-stop. If you show a random picture of either a ball, a bicycle, or a flower to the owner, then record what the bird says during the next two minutes, he's going to get a hit 100% of the time, even by chance.

Yes. Exactly. So the chance baseline is now 100%. This is a special situation where it's impossible to test if the bird scored higher than the chance baseline. But add in a fourth picture to the randomly selected pictures though, and a score above 75% on trials where the bird always said "ball bike flower", and sometimes a fourth word, would be evidence of communication.

This should be proof enough that the bird's baseline speech patterns alone (regardless of any psychic phenomenon) can obviously affect the statistics. You need to take it into account before you can point to anything as a possible anomaly.

Yes, the bird's baseline speech "affects the statistics". It changes the baselines we need to compare to for each subset of responses. And yes, it needs to be taken into account.

Luckily, it was. As I suspected, I believe the reason this study used uncommon statistical techniques (randomized permutation and bootstrap analyses) is that these take this unusual situation into account. By randomizing the specific set of responses given, the computer creates the chance baseline itself. Because the randomized responses also match up with the bird's multiple-guess format, this is taken into account. Out of all the randomized orders of guesses, the actual number of hits (or more) obtained in the real study were extremely rare. In other words, if chance alone were operating, that number of hits would almost never occur. Thus, it's unlikely that chance alone was operating. (Note: Don't quote me on this, as I'm not a total expert on these techniques - but don't dispute me either unless you have reason to!)

Originally Posted by Phronk View Post
To further illustrate why it is fair to throw out the other 90 trials, let's assume that only chance is operating. The bird is just babbling and is not influenced by the owner's thoughts at all.

On the first trial, the bird says "ball". What are the chances that he's right? Well, 1/3, because there's a 1/3 chance that the computer randomly selected the ball picture for the owner.

Wrong... The chance that you'll get a ball picture is 1/3. The chance that the bird will say "ball" cannot be calculated without knowing how often the bird says ball in general. Maybe he says ball every 1 minute on the minute, [...] during a 2 minute period after showing the picture, there is a 100% chance that the bird will say "ball" during that trial. That means in your post analysis, regardless of what the other trials say, you are going to see a 100% hit rate on the ball pictures.

Right. But we're not only looking at ball pictures. If we were, we'd hit the 100% baseline ceiling like in the example from before. In my original example, with a set of 3 pictures, if the bird said "ball" every minute, he'd get 100% on the ball pictures, 0% on the flower pictures, and 0% on the bike pictures. Overall, he'd get 33.33%. Exactly chance, and correctly, not interpreted as any kind of communication. All without knowing anything about the bird's baseline utterances.

So no, I'm not wrong. You say "the chances that the bird will say ball cannot be calculated without knowing how often the bird says ball in general". While true, it's beside the point. We're interested in when the bird says "ball" AND the ball picture was randomly selected. And that is something we can model perfectly without knowing what the bird says in general.

You are absolutely right when saying that we want to compare the actual results to what we would expect by chance alone. The only problem is that the experiment did not figure out what to expect by chance alone. I think I've sufficiently demonstrated this with my previous examples.

And I think I've sufficiently demonstrated that they did figure out what to expect by chance alone. Particularly important to the real study (vs. my example) is the randomized permutation analysis. I suggest looking up more about this technique if anyone is unfamiliar with it. I suspect the critics of this experiment did not do this, and thus their criticism was not entirely convincing (to me, at least). There may be biases in it, but to discover them would require digging deeper into the statistics.

I absolutely agree with this statement. Along the exact same reasoning, trials during which the bird did not respond with words matching to the owner's "psychic" signal are evidence that the stimulus does not exist. But most of this data was thrown out, apparently using the very excuse that the bird did not appear to get any psychic messages during that trial! If that's not bias, I don't know what is.

This again. I've already shown that looking at the subset of key words cannot possibly bias the results toward above-chance scoring. Even if you think including every trial would answer a question more directly (a different question, but still a question I guess), the significant results in the subset are still strong evidence for communication.

Besides, looking at it your way ends up giving evidence for communication too.

It all depends on the hypothesis being tested. If your hypothesis is that there are psychic messages being sent from owner to bird, then throwing out trials that are clearly evidence to the contrary is obviously biased. If, on the other hand, your hypothesis is that an autistic child will respond to stimulus X using method A more often than method B, then it is safe to throw out the trials where the child does not respond at all to stimulus X, because it doesn't say anything one way or another about the hypothesis (which is only interested in comparing responses).

See the difference? In the first case, every trial has important data that can answer the question being tested. We want to know, by looking at the bird's responses, if he's getting the stimulus at all. In the second case, only the trials where the child actually responds using either method A or method B are important to the question. If you were to do a study that hypothesizes autistic children will respond at all to stimulus X, then obviously throwing out trials where the child doesn't respond would be a ridiculous thing to do--just as it is in the parrot experiment.

Let's do a thought experiment.

If the bird experiment had added another variable, Method, like you propose for autistic children above, and found a difference in hit rates depending on method, would you then accept its results?

E.g. we find that when the owner is happy, the hit rate for happy pictures is higher than when the owner is sad, when the hit rate for sad pictures is higher. We throw out trials when the bird says things that are not related to either happy nor sad pictures.

We find a difference - the bird does better on happy pictures when the owner is sad, and the same for sad pictures and sad mood.

Do you say that this doesn't provide any evidence at all that the bird was reacting to the stimuli (the owner's photographs)? Was it biased because we threw out neutral words?

No. Of course not. If the bird is reacting differently to different "methods" (mood states), then he is certainly reacting! Throwing out neutral words made sense because they were irrelevant to the question at hand. But they didn't bias the results either way. The actual study done was even simpler, not including the extra variable, but threw out trials for the same type of reason.

In other words, if it doesn't bias results, and doesn't affect your conclusion, for autistic kids, why should it for parrots?

[Rudism]

I won't bother getting into another point/counter-point quote-fest... I'm not a mathematician, so I can't speak authoritatively on the methods they used. All I can say is that throwing 40% of the trials for no other reason than them not supporting your hypothesis is silly, and any amount of statistics you run the remaining results through to support your hypothesis is going to be overshadowed by that.

I also note that they threw out the word 'camera' from the list because the bird was saying it so often (due to cameras being used in the experiment). Wouldn't this effect have been cancelled out by the random permutation analysis which you claim does compare the hits against a baseline? Why throw it out if the bird's baseline speech patterns are already accounted for in the post-analysis? How do we know there aren't other words that the bird was using frequently enough to also warrant being thrown out (perhaps "flower," which accounted for almost half of the hits used in the final analysis)?

[Venom]

I listened to this episode today, and I have only one thing to say:

What a deluded woman.

[David Bailey]

Originally Posted by Venom

I listened to this episode today, and I have only one thing to say:

What a deluded woman.

LOL - we probably more or less agree on this one! I hope Alex will remember in future to ask the hard questions - even if it makes the interview less chummy!

David

[Phronk]

Originally Posted by Rudism

I won't bother getting into another point/counter-point quote-fest... I'm not a mathematician, so I can't speak authoritatively on the methods they used.

Fair enough. But I hope you trust my mathematical knowledge enough to at least be undecided about the results (as I am) rather than believing there is some obvious and fatal flaw with them.

All I can say is that throwing 40% of the trials for no other reason than them not supporting your hypothesis is silly, and any amount of statistics you run the remaining results through to support your hypothesis is going to be overshadowed by that.

Well, I thought I made it clear that there was a good reason for throwing them out; they're not relevant to the question being asked.

It's also worth noting that, if you are right and there is no communication going on at all, and thus any "hits" are due to chance alone, then there would be absolutely no difference in the chances of finding significant results with or without the thrown out trials. If it was properly conducted and analyzed, the chance of incorrectly concluding there was an effect if there wasn't one is exactly 0.05, or 5%.

So, I repeat, throwing out the trials introduced NO BIAS. If there was no effect, it wouldn't help him falsely find evidence for one. This may be counterintuitive, but unless someone can see a fault in my reasoning, it's a fact.

Think of it this way. A guy claims he's really good at predicting coin flips, but only when it's sunny outside. He predicts coins in a room with a window and is videotaped. The coin flipper goes on flipping no matter what, but sometimes it gets cloudy outside. Later, the videotape is examined, and trials where it was cloudy out are thrown out, because that wasn't the guy's claim.

If he really was guessing and couldn't predict the future, would it help him to throw out certain trials? Nope. The baseline of 0.5 remains the same because the flipper is always random. It didn't bias him at all. And we don't need to know base rates of his guessing (he could guess heads every time for all we care) or base rates of cloud-vs-sunny (all it does is determine the sample size of valid trials) to know this.

Would it be convincing if he did consistently guess more than 50% of the time? Yes. If anything, the smaller sample size would make it harder to find an effect if it really did exist. Significantly above-chance guessing at any subset that is chosen independently from hit rate is evidence of predicting the future.

I also note that they threw out the word 'camera' from the list because the bird was saying it so often (due to cameras being used in the experiment). Wouldn't this effect have been cancelled out by the random permutation analysis which you claim does compare the hits against a baseline? Why throw it out if the bird's baseline speech patterns are already accounted for in the post-analysis? How do we know there aren't other words that the bird was using frequently enough to also warrant being thrown out (perhaps "flower," which accounted for almost half of the hits used in the final analysis)?

I agree, that is slightly fishy. Especially if it was done after the fact, when hit rates were already known (e.g., they noticed that trials with "camera" being said were almost always misses). But the excuse that there were cameras in the room seems reasonable, and "signals" directly in the room would indeed only interfere with the psychic "signal" from a distance if it existed, adding noise to the effect they're looking at.

Still, if this was replicated, I'd expect them to exclude camera from the start.

[David Bailey]

It is interesting that not so many years ago scientists used to claim that parrot speech was pure mimicry - devoid of all understanding - simply something they had evolved to do to help them catch prey in the forest! Maybe more experiments and less theory is in order!

David

[Rudism]

Originally Posted by Phronk

Well, I thought I made it clear that there was a good reason for throwing them out; they're not relevant to the question being asked.

What exactly is the question being asked? This is from the abstract of the published paper:

Aimée Morgana noticed that her language-using African Grey parrot, N'kisi, often seemed to respond to her thoughts and intentions in a seemingly telepathic manner. We set up a series of trials to test whether this apparent telepathic ability would be expressed in formal tests...

The parrot "often" seems to respond. They want to test if this is true. Seems to me that any trials where the bird doesn't respond is directly relevant to the question being asked. The question has been redefined after the fact because the actual data didn't support the original hypothesis.

Originally Posted by Phronk

Think of it this way. A guy claims he's really good at predicting coin flips, but only when it's sunny outside. He predicts coins in a room with a window and is videotaped. The coin flipper goes on flipping no matter what, but sometimes it gets cloudy outside. Later, the videotape is examined, and trials where it was cloudy out are thrown out, because that wasn't the guy's claim.

There's no real equivalent to the "cloudy out" part of this analogy in the parrot example, but I will grant you that in this case it wouldn't make a difference. Also, the guy making guesses should be allowed to make more than one guess on trials when he feels like it (which, in this case, would always result in a hit, since there's only two possible guesses he can make). In this case, we would expect an overall hit rate of greater that 50%, since the "heads tails" guesses would skew it in that direction. You're claim is that the random permutation analysis would allow us to say whether or not the results are significant even with the "heads tails" guesses factored in. I'm not so sure about this, but I'm curious enough that I'm going to try to get a copy of the actual paper and delve into some math myself to see what they actually did.

[Phronk]

Originally Posted by Rudism

What exactly is the question being asked? This is from the abstract of the published paper:

The parrot "often" seems to respond. They want to test if this is true. Seems to me that any trials where the bird doesn't respond is directly relevant to the question being asked. The question has been redefined after the fact because the actual data didn't support the original hypothesis.

I took the question being asked as "when the parrot says one (or more) of this set of of words from his vocabulary, does it tend to match up with pictures representing that set of words?"

I do see your point; if this psychic signal was so strong and obvious, maybe the bird should drop everything else he's saying and say the word. But as I've been saying, I think it's a cleaner test of the effect to look for it in situations when it's likely to be strongest (i.e., when he's speaking intelligible words from his vocabulary). In essence, it enhances the signal to noise ratio. As I think I've shown, it wouldn't help detect a signal that wasn't there to begin with, but it can enhance one that is already there.

Also, I don't know if it was an after-the-fact thing, as you guess. It's always hard to tell with written reports that necessarily are written after the fact, but I think a clever researcher would specify this limited analysis to start with, before the study was even run. You also say it was done after the fact because the full data set didn't support the hypothesis. This is demonstrably false. Again: The full data set DID support the hypothesis.

There's no real equivalent to the "cloudy out" part of this analogy in the parrot example, but I will grant you that in this case it wouldn't make a difference. Also, the guy making guesses should be allowed to make more than one guess on trials when he feels like it (which, in this case, would always result in a hit, since there's only two possible guesses he can make). In this case, we would expect an overall hit rate of greater that 50%, since the "heads tails" guesses would skew it in that direction.

True, multiple guesses would make it closer to the actual study, but wouldn't be practical in my simple dichotomous-response example (since the baseline in any trials he guessed twice would be 100%, and thus useless). Still, the general principle is there.

The equivalent to the "when it is cloudy" part is "when the bird is talking intelligibly." The claim is that when the bird is talking, he's talking about the owner's thoughts.

It'd be interesting to test your assumptions too - that the bird should start talking when the owner is "thinking at him". But again, that's not what I interpreted the question to be.

You're claim is that the random permutation analysis would allow us to say whether or not the results are significant even with the "heads tails" guesses factored in. I'm not so sure about this, but I'm curious enough that I'm going to try to get a copy of the actual paper and delve into some math myself to see what they actually did.

Awesome. I'm going to keep this study in mind as I dig further into these types of analyses, too.

Another thing I haven't mentioned yet is that they threw some trials out from the random permutations, too: trials in which the same word was said more than once. Intuitively, this wouldn't seem to bias the results, and is pretty much necessary to get the proper baseline, but I haven't thought about it enough to say for sure.

I do hope one of us finds a rational explanation here. Because birds reading my mind = scary as hell.

[jacob]

Just saw this article about a research on dogs:
Research indicates dogs have some ability to read minds (11/7/2007)

Canterbury University psychology student Michelle Maginnity has just completed a masters research project looking at whether the domestic dog has a theory of mind - that is, whether they can think about the thoughts and feelings of self and others.

She said after carrying out a range of experiments which tested the cognitive skills of dogs, she believed they were not only sensitive to human cues, but also had the ability to think about what their human companions may be thinking.

"So, in a way, dogs may be able to read minds," she said.

Michelle's research involved testing the social-cognitive skills of 16 dogs, some pure bred and others of mixed breed, in a food-finding task. In four different experiments the dogs had to decide where the food was hidden by following cues from people who either did or didn't know where the food was.