Dr Rupert Sheldrake
Is it Real? Psychic Animals
National Geographic Channel UK,
Various dates 25 July 7 Oct 2005
Ofcom has upheld in part this complaint of unfair treatment.
Dr Rupert Sheldrake was a participant in the programme and provided details about his study into the case of a parrot that he reported was able to respond telepathically to his owner. A sceptic replicated Dr Sheldrake's test with a different parrot and concluded that Dr Sheldrake's analysis and interpretation of Dr Sheldrake's original test results were flawed.
Dr Sheldrake complained of unfair treatment in that: the programme presented his work in a false and misleading way; the programme makers did not honour their assurances that the programme would be presented in a fair and unbiased way; and, neither he nor any other qualified scientist had been offered an opportunity to respond to the sceptic's criticism.
Ofcom found the following:
1. The programme did not make false claims about Dr Sheldrake's research, rather a critique of his analytical approach was offered. This critical point of view was acceptable given both the expectation that scientific research would and should be subjected to examination and that Dr Sheldrake's scientific referees had raised various queries about the way he analysed his research data. 2. The lack of a balancing view in the programme as broadcast led to the breaking of the guarantee given to Dr Sheldrake regarding the content of the programme. This resulted in unfairness to Dr Sheldrake. 3. The programme makers' failure to give Dr Sheldrake an opportunity to respond to what would amount to a damaging critique of his research resulted in unfairness to Dr Sheldrake.
This programme examined: the reported ability of animals to predict disasters; whether pets have telepathic connections with their owners; and whether pets can be psychic. Specifically, the programme questioned whether any such abilities were a "sixth sense" or simply a natural super sensitivity that animals have to their surroundings.
Dr Rupert Sheldrake was a participant in the programme. He provided details about his study into the case of a parrot, called N'kisi, and reported that the animal was able to respond telepathically to its owner. This experiment had involved placing N'kisi and his owner in separate rooms. The owner then looked at pictures of objects (which N'kisi had identified in the past) for a set amount of time. Every time that N'kisi was able to say the name of an object, at the same time that the owner was looking at a picture of the object in the other room, a "hit" would be recorded. The premise of the experiment was that the more hits recorded in the trials, the more telepathic N'kisi proved to be. The full details and results of Dr Sheldrake's experiment were published as a research paper in the Journal of Scientific Exploration ("the Journal") and this research paper was referred to in the programme.
The programme provided a summary of Dr Sheldrake's experiment in the programme and later placed those tests under the scrutiny of a sceptic, Tony Youens. Mr Youens attempted to replicate Dr Sheldrake's test with a different parrot called Spaulding and concluded that Dr Sheldrake's analysis and interpretation of Dr Sheldrake's original test results were flawed.
1. Mr Youens claimed Dr Sheldrake skewed N'kisi's test results by only counting the trials when N'Kisi actually spoke. This criticism was accompanied by the quote "they [those trials where the parrot did not speak] were irrelevant to the analysis" which was taken from Dr Sheldrake's research paper.
2. Mr Youens also claimed that Dr Sheldrake skewed N'kisi's test results by not counting the trials if the picture cards showed an object which N'kisi rarely said. Similarly, the criticism was accompanied by the quote "exclude the trials involving those images" which was taken from Dr Sheldrake's research paper.
3. Mr Youens claimed that if Spaulding's test results were analysed in the way Dr Sheldrake had analysed N'kisi's, Spaulding would appear to be telepathic also.
Dr Rupert Sheldrake complained that he was treated unfairly in the programme as broadcast.
Dr Sheldrake's case
In summary, Dr Sheldrake complained that:
a) the programme presented his research material in a false and misleading way. Specifically it was claimed in the programme that:
i) Dr Sheldrake's results were skewed because in his test he only counted trials when parrot actually responded to the stimuli (as opposed to making no noise);
ii) Dr Sheldrake's results were further skewed because in his test he discounted those trials which required the parrot to say a rarely used word.
iii) When Dr Sheldrake's methods of analysis were applied in the programme's test, the programme's parrot appeared telepathic with a success rate of 27%;
b) the programme makers gave assurances that Dr Sheldrake's work would be presented fairly and without bias; which they did not fulfil; and,
c) the programme makers did not offer Dr Sheldrake, nor any other qualified scientist, an opportunity to respond to Mr Youens' claims.
National Geographic Channel - UK's (National Geographic) case
a) National Geograhic and the programme maker National Geographic Television & Film maintained that Dr Sheldrake's research and material was not presented in a false and misleading way. The broadcaster explained that more than a dozen scientists and experts, many of whom did not appear on-screen, had been consulted in the development of the programme and their critique included within it.
It was noted that Dr Sheldrake's original published paper had been reviewed by two experts. One expert expressed repeated concern about the statistical analysis in the paper, and concluded, "I do not believe that this experiment provides any evidence supporting the claim of telepathy". In response to Dr Sheldrake's specific complaints, National Geographic responded as follows:
i) With regard to the programme's critique of Dr Sheldrake's decision to exclude trials when N'kisi did not speak, the broadcaster said their experts provided strongly supported arguments that it was wrong not to include these instances. It was suggested that if such data was removed, the outcome of the test could be determined by the chattiness of the bird on a particular day. National Geographic denied that the quote "they were irrelevant to the analysis" that was used in the programme (taken from Dr Sheldrake's research paper), had been used out of context. It was maintained that the quote was used as it was meant in the research paper.
ii) In relation to the exclusion of rarely used words from the results National Geographic said experts had again been consulted. For example an expert statistician reviewed the paper and made a number of critiques which included:
"The experimenter has complete control to pick unambiguous targets, and that is exactly what should have been done...if there are more images that correspond with commonly used key words, the bird has an advantage...the idea of throwing out some trials has no justification either the bird got it, or not".
Programme makers also denied that they used the quote "exclude the trials involving those images" from Dr Sheldrake's research paper in an unfair or misleading way. They explained it had been drawn from a section of the research paper, where Dr Sheldrake outlined a better method of analysis and had been used in context.
iii) Programme makers said that they made all efforts in good faith to replicate Dr Sheldrake's experiment with their parrot Spaulding. National Geographic maintained that if the analysis methods used by Dr Sheldrake during his test with N'kisi were applied to the test with Spaulding, Spaulding appeared telepathic with a score of only 29% (as stated in the programme). This score was calculated by the exclusion of trials in which Spaulding did not speak and the removal of trials where Spaulding was required to state a rarely used word; the methods originally used by Dr Sheldrake.
b) National Geographic said that programme makers had told Dr Sheldrake that the programme would not be biased in either direction and believed this assurance had been met. The broadcaster maintained that Dr Sheldrake's research and material had been accurately presented in the context of scientific enquiry. The material had then been critiqued and tested in a fair, accurate and non-biased manner. The producer of the programme explained to Dr Sheldrake in a letter to him that where necessary the "flip side of the coin" would be presented. National Geographic said that this was exactly what the programme achieved. The programme had made no final conclusions but left them to be drawn by the individual viewer.
c) National Geographic said that Dr Sheldrake's response to the sceptical point of view was included in the programme and referred to the following comment made by the complainant in the programme: "Totally to ignore the animals seems to me dogmatic, not scientific. And I'm more interested in dogs than dogmas".
In addition Dr Sheldrake and his colleague had been invited to participate in the contra-experimentation, however both declined. National Geographic noted that the programme makers consulted many "other qualified scientists" in developing the programme, all of whom supported various aspects of the critique presented in the programme.
Dr Sheldrake's response to National Geographic's statement
a) Dr Sheldrake rejected the claim that the programme had presented his work in a fair way. Dr Sheldrake said that Tony Youens, together with the narrator, tried to give the impression that Dr Sheldrake had manipulated the results of his experiment by omitting data and carrying out calculations designed to magnify the significance of the findings. The programme implied that once Tony Youens had found 'holes' in Dr Sheldrake's experiment, the positive results and significance of the research paper itself disappeared also, which was false.
Dr Sheldrake said the programme's treatment of his decision to remove trials when N'kisi did not respond, was flawed for two reasons:
i) The programme did not understand that the decision not to include these trials was in line with established practice in mainstream research with animals, young children and autistic people. Dr Sheldrake explained that analysis is performed in this way, to take into consideration the subject's limited attention span and inability to know that they are being tested.
ii) Notwithstanding the first reason, Dr Sheldrake said the programme completely ignored a key finding of one of the paper's reviewers. This reviewer, included at the end of the paper, directly questioned and tested the effect that the removal of non-response trials had had on the results. The reviewer found that if the non-response trials were included, the results "differed only trivially". Therefore it was false for the programme to imply that by omitting these trials the results would have altered.
Dr Sheldrake said the programme implied that by removing the trials, where rarely used words were used, from the analysis of test results he increased the probability that N'kisi would appear telepathic.
However, Dr Sheldrake said that, as his paper had explained, by removing such trials the opposite occurred: the removal of such trials "made the result slightly less significant, rather than more so". Dr Sheldrake said the programme failed to explain that regardless of which methods of analysis were used, the experiment's results remained significantly above the level of chance.
Dr Sheldrake maintained that the test conducted by programme makers was flawed, therefore making a comparison between the two tests unscientific. However notwithstanding such flaws, Dr Sheldrake said that the programme's attempts to apply his methods of analysis were misleading for the following reasons:
i) Dr Sheldrake's conclusions were not based on 'percentage hit rates', as used by programme makers. Rather the conclusions were based on standard kinds of statistical probability analysis including randomised permutation analysis.
ii) The programme implied that N'kisi's success in telepathy tests was a result of data manipulation rather than due to any genuine ability of N'kisi. The programme made it appear to viewers that Dr Sheldrake had omitted or massaged data to get the desired result, regardless of what the facts indicated. Dr Sheldrake said his results were analysed in several alternative ways and the significance of the results were not dependent of the type of analysis used.
b) Dr Sheldrake rejected National Geographic's claims that the programme was "fair, accurate and non-biased" and disputed that the "flip side of the coin" had been presented. Dr Sheldrake noted that the choice of experts used by programme makers ensured that the programme did not present both sides of the argument fairly.
c) Dr Sheldrake maintained he had been given no chance to reply to Mr Youens' critique of his research, or to comment on the programme's experiment. Dr Sheldrake said the quote referred to by NCG-UK as being Dr Sheldrake's response to the sceptical point of view was used out of context and had been filmed before the experiment took place.
National Geographic's second statement
The broadcaster maintained that Dr Sheldrake was not treated unfairly in the programme as broadcast. Programme makers said the programme's content and context had been clearly explained to Dr Sheldrake, prior to his participation, and that Dr Sheldrake was able to clearly represent his views at various stages during the programme. Further, the programme made no allegations against Dr Sheldrake of wrongdoing or incompetence. National Geographic said their programme was produced in a thoroughly professional manner and was fair and balanced
Ofcom's statutory duties include the application, in the case of all television and radio services, of standards which provide adequate protection to members of the public and all other persons from unfair treatment in programmes and unwarrantable infringement of privacy in and in the making of programmes included in such services.
In carrying out its duties, Ofcom has regard to the need to secure that the application of these standards is in the manner that best guarantees an appropriate level of freedom of expression. Ofcom is also obliged to have regard in all cases, to the principles under which regulatory activities should be transparent, accountable, proportionate and consistent and targeted only at cases in which action is needed.
Ofcom acknowledged that within the field of scientific study and research, discussion over different methods of statistical analysis is common. Indeed such discussion and debate are essential for the formulation of robust, statistically sound scientific theories and findings. It is not for Ofcom to adjudicate on whether any particular type of statistical analysis is better than another, but rather whether a person has been treated unfairly in a programme. Accordingly Ofcom considered it was likely that within any field of scientific study, and in particular for emerging fields of study, the opportunity for debate over methods of statistical analysis would be expected and warranted.
However, Ofcom noted that if a programme alleges wrongdoing or incompetence or makes other significant allegations, those concerned should normally be given an appropriate and timely opportunity to respond.
a) Dr Sheldrake complained that the programme made three false claims about his work. Ofcom did not consider the programme made false claims with regard to its assessment of Dr Sheldrake's work. Rather it is Ofcom's opinion, that the programme provided a critique of Dr Sheldrake's methods of analysis, which was warranted given the acknowledged differing opinions of what was the best form of analysis for this type of experiment. When coming to this conclusion, Ofcom took into consideration the varying expert opinions provided in Dr Sheldrake's paper and those offered by the broadcaster. In particular Ofcom noted that the editor of "the journal" made the following acknowledgment:
"This article is another instance of your Editor's difficulties where research protocols and statistical inference are questioned...I want to express publicly our deep indebtedness to reviewers who have time and again spent much time and effort in clarifying issues, stimulating authors to refine their presentations and informing readers of the various views that can be legitimately taken on some of these matters." [emphasis added]
Therefore, Ofcom found that the programme did not make false claims about Dr Sheldrake's research; rather a critical point of view was offered. This critical point of view was acceptable given the various views taken on some of the matters raised in Dr Sheldrake's research.
Ofcom found no unfairness in this respect.
b) Dr Sheldrake complained that the programme makers gave assurances that his work would be presented fairly and without bias, which they did not fulfil. Ofcom noted that both broadcaster and complainant offered correspondence which confirmed that such an assurance had been given by programme makers to Dr Sheldrake.
"Being National Geographic, and having a very strict policy of balanced reporting, we cannot be biased in either direction. It is our job to present the work being done and where deemed necessary and in all fairness, we will often include the flip side of the coin. I absolutely have no intention of putting anyone in an unfair, uncomfortable position or making anyone look silly. My goal is to present science."
Ofcom's Broadcasting Code outlines that guarantees given to contributors for example relating to the content of a programme, confidentiality or anonymity, should normally be honoured.
In the programme, general details of how Dr Sheldrake carried out his experiment with the parrot N'kisi were given. This was then followed by a critical appraisal by Mr Youens of the way in which Dr Sheldrake analysed the results contained within his experiments. As stated above, such a critical view is acceptable and in itself did not result in unfairness to Dr Sheldrake. However, Ofcom noted that at no point were the criticisms expressed by Mr Youens countered or balanced by a response by Dr Sheldrake (or by another scientist of opposing opinion).
It is Ofcom's opinion that in order for the programme makers to meet the guarantee of unbiased and fair reporting (as noted in the email extract above), it was necessary for the programme makers to offer an alternative view to the critique given by Mr Youens. Ofcom concluded that the lack of such an opposing view in the programme as broadcast led to the breaking of the guarantee given to Dr Sheldrake regarding the content of the programme. This resulted in unfairness to Dr Sheldrake. Ofcom has upheld this part of the complaint.
c) Dr Sheldrake complained that the programme did not offer him or any other qualified scientist an opportunity to respond to Mr Youen's claims, which resulted in unfairness. As previously noted, if a programme alleges wrongdoing or incompetence or makes other significant allegations, those concerned should normally be given an appropriate and timely opportunity to respond. Accordingly, Ofcom first considered whether the programme made any such allegations. In this respect, Ofcom concluded that the programme's critique of Dr Sheldrake's work was capable of adversely affecting the regard in which Dr Sheldrake's work was held which in turn drew into question Dr Sheldrake's professional credentials. As such, Ofcom considered that in order for the programme not to be unfair to Dr Sheldrake, programme makers should have given Dr Sheldrake an opportunity to respond to the criticisms contained in the programme concerning the conduct of his experiment and his interpretation of that experiment. Ofcom noted that though Dr Sheldrake had been asked to make a contribution to the programme on a number of occasions, at no time was he asked to comment on the specific criticisms of his research which were to be included in the programme. This failure to give Dr Sheldrake an opportunity to respond to what would amount to a damaging critique of his research resulted in unfairness to Dr Sheldrake. Ofcom has upheld this part of the complaint.
Ofcom has partly upheld Dr Sheldrake's complaint of unfairness in the programme as broadcast.
The Executive Fairness Group
5 June 2006