Teaching scientific knowledge doesn’t improve scientific reasoning

View Images

On Tuesday, I wrote a short essay on the rightful place of science in our society. As part of it, I argued that scientific knowledge is distinct from the scientific method – the latter gives people the tools with which to acquire the former. I also briefly argued that modern science education (at least in the UK) focuses too much on the knowledge and too little on the method. It is so blindsided by checklists of facts that it fails to instil the inquisitiveness, scepticism, critical thinking and respect for evidence that good science entails. Simply inhaling pieces of information won’t get the job done.

View Images

This assertion is beautifully supported by a simple new study that compared the performance of physics students in the USA and China. It was led by Lei Bao from Ohio State University who wanted to see if a student’s scientific reasoning skills were affected by their degree of scientific knowledge. Does filling young heads with facts and figures lead to a matching growth in their critical faculties?

Fortunately for Bao and his team of international researchers, a ready-made natural experiment had already been set up for them, in the education systems of China and the US. Both countries have very different science curricula leading to different levels of knowledge, but neither one explicitly teaches scientific reasoning in its schools. If greater knowledge leads to sharper reasoning, students from one country should have the edge in both areas. But that wasn’t the case.

In China, all students work to the same standard courses until the 12th grade (Year 13 in the UK). For physics, those run for five years, starting at the 8th grade. The curriculum is heavy on both algebra, conceptual understanding and problem-solving skills. In the US, physics education is more varied. It’s covered in general science courses, but only one in three students take it up as a subject in its own right, and even then, just for two years.

It should come as no surprise that test scores reflect these differences. Using two standard physics tests – one on mechanics and one on electricity and magnetism – Bao tested the knowledge of 5,760 freshmen. Each was about to start a physics degree at one of four American and three Chinese middle-ranking universities.

In both tests, the Chinese students outperformed their American rivals. On the mechanics test, the average Chinese score of 86% easily trounced the American average of 49%. On the electromagnetism test, the Americans scored an average of 27% (barely better than chance) while the Chinese still averaged a respectable 66%. The Chinese students were also less varied in their scores, as you might expect of a nation that has a uniform curriculum. In contrast, the greater variability of American education led to a much wider range of results.

View Images

So the Chinese students clearly have the edge in terms of scientific knowledge. But in terms of scientific reasoning, students from both countries performed almost identically.  Bao also asked them to sit Lawson’s Classroom Test of Scientific Reasoning – another standard test, but one designed to examine basic scientific skills. It doesn’t require much knowledge, but instead probes for skills like wielding deductive and inductive logic, controlling different variables and testing hypotheses.

Faced with these questions, the average scores of the American and Chinese students were practically identical (74.7% and 74.2% respectively), as were their distributions. These striking similarities suggest that students from both countries share matching scientific reasoning skills, despite their vastly different educational systems.

To Bao, it’s further evidence that modern science education “emphasizes factual recall over deep understanding of science reasoning”. Even if this fact-focused system is realised in a rigorous way over many years, as it is in China, it still doesn’t lead to vastly better scientific reasoning skills. And while Bao’s study only looked at physics but there’s reason to believe that the same would apply for other scientific disciplines. After all, both countries have similar differences in the way they teach biology and chemistry.

Now, you might argue that the fact that both sets of students still achieved respectable scores in the reasoning tests proves otherwise – surely if their education was bad at instilling reasoning skills, the average scores would be much lower? Well for a start, you might expect that the sample group – students who were already starting a university physics course – would have a natural edge in reasoning skills anyway.

But Bao also points out, “Their reasoning ability was developed through both formal education (or school education) and informal education (learning in real life).” His data show that different standards of education, leading to different levels of knowledge, don’t translate into different aptitudes for reasoning.

So if it what we teach doesn’t lead to better reasoning skills, perhaps it is how we teach it. That’s an idea Bao subscribes to. He advocates more “inquiry-based learning“, where teachers encourage students to discover things for themselves, acting more as a facilitator that guides the class rather than a vessel that pours knowledge into it. Obviously, this shouldn’t be introduced to the exclusion of knuckling down and cramming some basic facts in, but the two approaches should complement each other. If this concluding chapter feels a bit rushed, it’s because I freely admit that I have no experience in the educational sector – I’m more interested to hear the opinions of readers who are.

Reference: L. Bao, T. Cai, K. Koenig, K. Fang, J. Han, J. Wang, Q. Liu, L. Ding, L. Cui, Y. Luo, Y. Wang, L. Li, N. Wu (2009). PHYSICS: Learning and Scientific Reasoning Science, 323 (5914), 586-587 DOI: 10.1126/science.1167740

Image: Graphs from Science; classroom by P.Morgan

View Images