Scientific Papers on Experimenter Effects

Be More Active in Reports

Times Educational Supplement January 7, 2005
by Rupert Sheldrake
Full Text

Abstract
Why are many schoolchildren still told to write up their science reports in the passive, as though experiments happened of their own accord? Many scientists abandoned this convention years ago. Watson and Crick's famous paper on the structure of DNA in Nature was in the active voice in 1953.

Blind Research: Are the Hard Sciences Immune From Experimenter Effects?

Skeptic (2003), Vol 10, No 1
by Rupert Sheldrake
PDF

Abstract
In scientific research, as in everyday life, our beliefs and biases often influence how we observe and interpret the world. In experimental psychology and clinical research, this problem is widely recognized, which is why experiments in these subjects are often carried out under blind or double-blind conditions. There is solid experimental evidence that experimenters' attitudes and expectations can influence the outcome of experiments.

Personally speaking

New Scientist, July 19, 2001
by Rupert Sheldrake
Full Text PDF

Abstract
Most scientific journals accept papers in the active voice and some, including Nature, positively encourage it. When I surveyed the current issues of 55 journals in the physical and biological sciences I found only two that still required contributors to use the passive.

How Widely is Blind Assessment Used in Scientific Research?

Alternative Therapies 5(3), 88-91, May 1999
by Rupert Sheldrake
Full Text PDF

Abstract
In everyday life, as in scientific research, "our beliefs, desires and expectations can influence, often subconsciously, how we observe and interpret things", as a recent article in the Skeptical Inquirer expressed it.(note 1) In experimental psychology and clinical research, these principles are widely recognized, which is why experiments in these subjects are often carried out under blind or double-blind conditions. There is overwhelming experimental evidence that experimenters' attitudes and expectations can indeed influence the outcome of experiments.

Experimenter Effects in Scientific Research: How Widely are they Neglected?

Journal of Scientific Exploration 12, 73-78, 1998
by Rupert Sheldrake
Full Text PDF

Abstract
A survey of recent papers published in a range of scientific journals showed that the used of blind methodologies is very rare in the so-called hard sciences. In the physical sciences, no blind experiments were found among the 237 papers reviewed. In the biological sciences, there were 7 blind experiments out of 914 (0.8%). There was a higher proportion in the medical sciences, 6 out of 102 (5.9%), and in psychology and animal behaviour, 7 out of 143 (4.9%). By far the highest proportion (85.2%) was in parapsychology. A survey of science departments in 11 British Universities showed that blind methodologies are neither used nor taught in 22 out of 23 physics and chemistry departments, or in 14 out of 16 biochemistry and molecular biology departments. By contrast, blind methodologies are sometimes practised and taught in 4 out of 8 genetics departments, and in 6 out of 8 physiology departments. I propose a simple procedure that could be used to detect possible experimenter effects in any branch of science, by comparing the results of a given experiment conducted both under open and blind conditions.

Could Experimenter Effects Occur in the Physical and Biological Sciences?

Skeptical Inquirer 22(3), 57-58 May / June 1998
by Rupert Sheldrake
Full Text PDF

Abstract
Probably most skeptics would agree with Michael Mussachia (SI Nov/Dec 1995) that "our beliefs, desires and expectations can influence, often subconsciously, how we observe and interpret things". In psychology and clinical medicine these principles are widely recognized, which is why experiments in these subjects are often carried out under blind or double-blind conditions.

Related Research by Others
Rupert Sheldrake and the Objectivity of Science

Skeptical Inquirer, 1999, vol 23 no 5
by Richard Wisemand and Caroline Watt
PDF

Abstract
Controversial biologist Rupert Sheldrake has recently published surveys suggesting that much of the current research in science may suffer from an important methodological problem that could seriously challenge the validity of many scientific findings. This article sets in motion a project designed to assess the impact of Sheldrake's provocative findings.

Skeptical Inquirer 22(3), 57-58 May / June 1998
by Rupert Sheldrake
Full Text PDF

Abstract
Probably most skeptics would agree with Michael Mussachia (SI Nov/Dec 1995) that "our beliefs, desires and expectations can influence, often subconsciously, how we observe and interpret things". In psychology and clinical medicine these principles are widely recognized, which is why experiments in these subjects are often carried out under blind or double-blind conditions.

Skeptical Inquirer 22(3), 57-58 May / June 1998
by Rupert Sheldrake

Probably most skeptics would agree with Michael Mussachia (SI Nov/Dec 1995) that "our beliefs, desires and expectations can influence, often subconsciously, how we observe and interpret things". In psychology and clinical medicine these principles are widely recognized, which is why experiments in these subjects are often carried out under blind or double-blind conditions.

In a double-blind clinical trial, for example, some patients are given tablets of a drug and others are given similar-looking placebo tablets, pharmacologically inert. Neither clinicians nor patients know who gets what.

In such experiments, the largest placebo effects usually occur in trials in which both patients and physicians believe a powerful new treatment is being tested (Roberts et al., 1993). The inert tablets tend to work like the treatment being studied, and can even induce its characteristic side-effects (White et al, 1985). Likewise, experimenter expectancy effects are well know in experimental psychology, and also show up in experiments on animal behaviour (Rosenthal, 1976).

How widespread are experimenter expectancy effects in other branches of science? No one seems to know. I have attempted to quantify the attention paid to experimenter effects in different fields of science by means of two surveys.

The first survey was of experimental papers recently published in leading scientific journals, including Nature and the Proceedings of the National Academy of Sciences (Sheldrake, 1998). In the physical sciences, no blind experiments were found among the 237 papers reviewed. In the biological sciences, there were 7 blind experiments out of 914 (0.8%); in the medical sciences, 6 out of 102 (5.9%); in psychology and animal behavior, 7 out of 143 (4.9%). By far the highest proportion (but the smallest sample) was in parapsychology, 23 out of 27 (85.2%).

The second survey was of science departments at 11 British Universities (including Oxford, Cambridge, London and Edinburgh). It confirmed that blind procedures are rare in most branches of the physical and biological sciences. They are neither used nor taught in 22 out of 23 physics and chemistry departments, or in 14 out of 16 biochemistry and molecular biology departments (Sheldrake, 1998). By contrast, blind methodologies are practised and taught in 4 out of 8 genetics departments, and in 6 out of 8 physiology departments. Even so, in most of these departments they are used occasionally rather than routinely, and are mentioned only briefly in lectures.

Only in exceptional cases are blind techniques used routinely. This survey revealed three examples. All three involved commercial contracts, according to which the university scientists were required to analyze or evaluate coded samples without knowing their identity.

When academic scientists were interviewed for this survey, some did not know what was meant by the phrase "blind methodology". Most were aware of blind techniques, but thought that they were necessary only in clinical research or psychology. They believed that their principal purpose was to avoid biases introduced by human subjects, rather than by experimenters. The commonest view expressed by physical and biological scientists was that blind methodologies are unnecessary in their fields because "nature itself is blind", as one professor put it. Some admitted the theoretical possibility of bias by experimenters, but thought it of no importance in practice. And one chemist added, "Science is difficult enough as it is without making it even harder by not knowing what you are working on."

The assumption by most "hard" scientists that blind techniques are unnecessary in their own field is so fundamental that it deserves to be tested empirically (Sheldrake, 1994). Not just in psychology and medicine but in all branches of experimental science we can ask: Can the expectations of experimenters introduce a bias, conscious or unconscious, into the way they carry out their procedures, make observations or select data?

I suggest the following empirical investigation. Take a typical experiment involving a test sample and a control, for example the comparison of an inhibited enzyme with an uninhibited control in a biochemical experiment. Then carry out the experiment both under open conditions, and also under blind conditions, with the samples labelled A and B. In student practical classes, for instance, half the class would do the experiment blind, while the other half would, as usual, know which sample is which.

If such tests show no significant differences, then for the first time there will be evidence that blind techniques are unnecessary. On the other hand, significant differences between results under blind and open conditions would reveal the existence of experimenter effects. Further research would then be needed to find out how the experimenters' expectations were influencing the data.

The more independent investigations, the better. It cannot be healthy for the supposed objectivity of regular science to rest on untested assumptions. Perhaps it will turn out, after all, that "hard" scientists need not bother with blind techniques. They may indeed be exceptions to the principle that "our beliefs, desires and expectations can influence, often subconsciously, how we observe and interpret things." On the other hand they may be like everybody else, including experimenters in psychology, parapsychology and medicine. Who knows?

References:

Roberts, A.H., Kewman, D.G., Mercier, L. & Hovell, H. 1993. The power of nonspecific effects in healing: implications for psychosocial and biological treatments. Clinical Psychology Review 13: 375.

Rosenthal, R. 1976. Experimenter Effects in Behavioral Research. New York: John Wiley.

Mussachia, M. 1995. Objectivity and repeatability in science. Skeptical Inquirer 19 (6): 33-5, 56.

Sheldrake, R. 1994. Seven Experiments that Could Change the World , Chapter 7. London: Fourth Estate.

Sheldrake, R. 1998. Experimenter effects in scientific research: How widely are they neglected? Journal of Scientific Exploration 12: 1-6.

White, L., Tursky, B. & Schwartz, G. (eds) 1985. Placebo: Theory, Research and Mechanisms. New York: Guilford Press.

New Scientist, July 19, 2001
by Rupert Sheldrake
Full Text PDF

Abstract
Most scientific journals accept papers in the active voice and some, including Nature, positively encourage it. When I surveyed the current issues of 55 journals in the physical and biological sciences I found only two that still required contributors to use the passive.

New Scientist, July 19, 2001
by Rupert Sheldrake

Why are school children still told to write up their science in the passive, as though the experiments happen of their own accord, asks Rupert Sheldrake.

"The test tube was carefully smelt." I was astonished to read this sentence in my 11-year-old son's science notebook. At primary school his science reports had been lively and vivid. But when he moved to secondary school they became stilted and passive. This was no accident. His teachers told him to write this way.

When I was at school, my science teachers made me write in the passive voice, but I had no idea it was still going on. Ever since I was a graduate student at Cambridge, I have thought the active voice -"I did" - far more appropriate in scientific writing than the passive -"it was done". Experiments do not mysteriously unfold in front of impersonal observers. People do science, and to portray it as a human activity is not to diminish it but to show it as it is.

The passive style is not only misleading, it is also alienating. A young medical student told me "it felt strange at first" when a lecturer asked her to write her reports in the active. "But then it felt liberating," she said. "Suddenly I could be myself again, after years pretending I wasn't there."

Recently I asked Frank Chennell, the co-ordinator of the Norfolk Teacher-Scientist Network (TSN), if he could find out how local teachers and scientists thought children should write science reports. Most teachers agreed that, in line with the national curriculum, younger children should adopt a direct style. But some believed that older pupils should use the passive. Most local scientists favoured the passive for research papers.

When Lord May, the president of the Royal Society, read the results of this survey in the TSN newsletter, he said he was "horrified" that the Norfolk scientists favoured the passive. "I would put my own view so strongly as to say that, these days, the use of the passive voice in a research paper is the hallmark of second-rate work," he says. "In the long run, more authority is conferred by the direct approach than by the pedantic pretence that some impersonal force is performing the research."

I soon found that May's strong views are shared by many other eminent scientists, including the Astronomer Royal, Sir Martin Rees. Bruce Alberts, president of the US National Academy of Sciences, has said he strongly favours the active voice.

Most scientific journals accept papers in the active voice and some, including Nature, positively encourage it. When I surveyed the current issues of 55 journals in the physical and biological sciences I found only two that still required contributors to use the passive.

As far as I can tell, the passive style didn't become fashionable in science until the end of the 19th century. It was meant to make science seem more objective, impersonal and professional. Before that, scientists generally used the active voice. Isaac Newton and Charles Darwin certainly did. The heyday of the passive in scientific literature was the half-century from 1920 to 1970. But while leading scientists have abandoned this convention, many science teachers still insist on it.

To find out more, I contacted the heads of science in 262 secondary schools: 212 state-maintained schools in Devon, Greater London (Camden, Ealing and neighbouring boroughs), Greater Manchester (Rochdale and Bury) and Nottinghamshire, and a random sample of 50 independent schools. I received replies from 172 of them. Overall, 45 per cent of the schools said they encouraged the use of the active voice, while 42 per cent said they encouraged the passive. The remaining 13 per cent had no preference.

There was a significant difference between state-maintained and independent schools: 58 per cent of the independent schools I surveyed encouraged the use of the passive, compared with 37 per cent of state schools. Geographically, the proportion of passive-inclined state schools ranged from 30 per cent in Devon to 41 per cent in London and Greater Manchester. Some of those teachers who taught using the active were enthusiastic advocates. Others said they used the active voice out of necessity, and one head of science in an inner-city comprehensive commented: "We're lucky to get them to write anything at all. It would be difficult to persuade students to write in a style so very different from normal speech." He implied that more state schools would use the passive if they could.

Other teachers promote the passive because they think examination boards require it. There is some truth in this. Of the three examination boards for England, two encourage the use of the passive for sixth form exams. The Qualifications and Curriculum Authority (QCA), the government's guardian of educational standards in England, has no official position on the matter.

Most of the teachers who encourage the passive voice say they are simply following convention. Clearly, they believe that leading scientists and journals still prefer it to the active. This is an outdated view. "Primary and secondary teachers should, without any reservation, be encouraging all their students to be writing in the active voice," says May. What would happen if the Royal Society officially endorsed the use of the active voice? Perhaps the QCA and the examination boards would follow suit. Then hundreds of thousands of science students could stop pretending that they were not really there during their experiments.

Science teachers in my survey who supported the active say it is "more natural". It "gives pupils ownership of their work" and "makes science more personal and pupils more involved". I agree. I believe the passive voice is alienating. It mystifies scientific practice and is ugly and cumbersome. The active is better at communicating what scientists actually do. Above all, it is more truthful.