Teaching in evolution

What a large replicated randomized controlled trial (surprisingly) tells us about how to teach evolution ⎮ 6 min read

Like Comment
Read the paper

Despite orthodoxy, active learning and human-centric subject matter do not make for the best way to teach evolution.


What is the best way to teach?

While this is a really simple question at first sight, in practice it is mighty hard to draw robust conclusions. Just as with vaccine and drug trials, large replicated randomised controlled trials (RCT) should be the gold standard. Implementing such trials in replicate, in a school environment is, however, as we know from experience, logistically  challenging. Busy teachers are hard to recruit and, once on board, may not always be compliant to the constraints of a trial. It was then fortunate for us that in 2014 a rare opportunity opened up when the UK government revised the National Curriculum for 10 to 11 year-olds (primary school), requiring the teaching of evolution and inheritance. As researchers interested in the teaching of evolution and genetics, our luck was in!

The opening of this ecological niche enabled many of the difficulties associated with teaching based RCTs to be overcome more easily. Primary school teachers are typically non-specialist and so might, we thought, be quite open to the provision of resources and training, deploying extensively prepared teaching plans with a series of lessons and complying with the protocol provided. We were not disappointed. We recruited in excess of 1000 students for the first tranche and over 1500 for a replicate study. Completion and compliance were excellent in both tranches.

The questions that we wanted to address were two-fold. 

First was whether the current use of active learning, as opposed to teacher-focused learning, enables improved understanding of, in our case, evolutionary concepts.  The second was whether relating the subject matter to the students (human-centric) was important, as prior literature suggesting the human-centric learning of evolution helps.

With this unusual opportunity, we were able to ask an additional and rarely asked question, thanks to fortuitous gaps in the teaching timetable. In a standard RCT, one typically compares say a drug treatment either against a placebo or against an alternative treatment in a pairwise test (A versus not A). However, in many cases the efficacy of any drug might be conditioned by drug-drug interactions. In a school context though, this could mean that the response to one lesson may be conditioned by the experience in another lesson. In education literature, such interactions are rarely considered, not least because the sample sizes needed for interaction-effect RCTs is considerable, usually requiring a minimum of four partitions not two. However, as lessons are taught in a series, which by definition are likely to lead to interactions, such interactions have the potential to mislead or overlook important misperceptions. With a considerable gap in the timetable for teaching the new elements of the curriculum, we took the opportunity to devise a unit of learning that enabled us to both test our core hypotheses and to ask whether there might be unsuspected interactions between teaching modes.

Developing a unit of learning

Following our earlier RCT showing that teaching inheritance/genetics prior to evolution1 leads to large gains in understanding (compared to the opposite order), we developed a four part lesson plan in which the 10 year old students were first taught inheritance (lesson 1), then natural selection (lesson 2), followed by the time line of evolution (lesson 3), and finished with a lesson on homology and common ancestry (lesson 4). All students attended the same lessons 1 and 3. For lesson two, students participated in an active learning game (hunting for moths, some camouflaged, some not) or the lesson adopted a teacher-centred story book approach. For lesson 4, they all completed a salt dough modelling exercise, either focused on the human/mammalian pentadactyl limb (using pet animal examples they could emotionally engage with) or on animals they could not develop an emotional bond with (trilobites). This created four schemes of work in a 2 x 2 structure, the classic design that drug-drug interaction studies follow. 

Example of lesson resources describing the trilobite anatomy.

With more than half an eye on implementation, the costs of the four lessons were kept extremely low (we estimate about £0.05 to £0.1 or $0.1 per student). To provide the trial with a decent scale and to help identify whether the lessons worked when taught by real teachers (rather than researchers), the classes were presented by the teachers (whose cooperation we are extremely grateful for).

To solve the problem of mass testing (rather than detailed qualitative analysis) we adapted our quantitative tool2 to enable fast and easy in-class delivery. It is an effective age-appropriate tool and freely available. Students were tested before and after the series of lessons. A more limited number had a third, follow-up assessment to test for longer-term retention of evolution concepts.

The results

We recently reported the results of this replicate RCT in teaching modes3. The results both appear to challenge educational orthodoxy and suggest that current experimental designs might easily lead to misleading results by failing to allow for interaction effects. Specifically, teacher-centric non-human focused teaching proved optimal, owing to replicable interaction effects.

Reassuringly, all schemes of work improved understanding of evolution concepts with effect sizes high enough for implementation. These results also applied (unusually in our experience) to the lower ability students. We also, however, found significant differences between the four schemes of work. Expectations were clear as to which should be best and worst.  If active human-centred learning is optimal, then the lessons with the moth game and the pentadactyly limb should be best, while the lessons with the story book and trilobites the worst. It was surprising to us the one predicted to be the worst was in fact the best, in replicate. 

Interestingly, this couldn’t have been predicted if we simply did the usual pairwise analysis (human-centric v not human centric on the one hand and active learning versus teacher centred on the other) as results were variable and largely not replicable. What was however, replicable, was a large interaction effect: teaching with the story book and the trilobites combined boosts educational gain beyond the expectations derived from considering both in isolation. This result is comparative to a drug test, in which a given drug doesn’t work unless prescribed along with another drug.  It is a troubling result, as the data suggests standard pairwise tests (A versus not A) have the power to mislead if the tests are not performed as part of a series.

Issues to be aware of

Other than this unexpected result, the analysis threw up a number of other cautionary reminders, most notably regarding the importance of replication. We tested pupil, teacher and school level predictors of student performance and many results from the initial trial failed in the replication phase. Some effects that were strongly significant in the first tranche were not replicable. For example, in the first tranche the story book approach was significantly better than the active learning alternative (at P =1.8 x 10-5), while in the replicate tranche the effect was in the opposite direction, although not significantly so. This suggests that reliance on even, well-designed tests without replication should be treated with a pinch of salt. It also cautions that a large sample size is no panacea in and of itself.

Aside from replication of the interaction effects of the success of the teaching programs and of the optimality of the scheme expected to be least optimal, we additionally determined student responses to teaching, by replicating a role for student ability (judged by teachers) and teacher confidence issues. We have implemented a free Massive Open Online Course to specifically address the latter, structured around the “genetics first” approach.

Understanding and Teaching Evolution MOOC (www.futurelearn.com/courses/understanding-evolution)

As many problems are raised by our analysis as it solves, and while it highlights interaction effects, the causes of these are poorly understood, even though we could rule out several explanations through the design of our experiment and the number of parameters studied. We are also aware that teaching evolution can be especially difficult because it requires replacing some notions with others, rather than simply adding novel material and concepts. Whether our lessons from teaching evolution apply more widely remains to be seen. These issues aside, we hope the resources we have tested and shown to work very well, can now be adopted. 

While the study challenges the primacy of human centric active learning, what the study doesn’t question is the importance of engagement for effective learning. Qualitative follow up analysis indicated strong engagement with stories and with fossils. It seems likely then there is more than one route to effective engagement. Perhaps we are just rediscovering the wheel in thinking that effective teaching starts with good engagement.

References

1 Mead, R., Hejmadi, M. & Hurst, L. D. Teaching genetics prior to teaching evolution improves evolution understanding but not acceptance. PLoS Biol. 15, e2002255, doi:10.1371/journal.pbio.2002255 (2017).

2 Buchan, L. L., Hejmadi, M. V. & Hurst, L. D. in Evolution Education Re-considered: Understanding What Works  (eds Ute Harms & Michael J. Reiss)  21-40 (Springer International Publishing, 2019).

3 Buchan, L., Hejmadi, M., Abrahams, L. & Hurst, L. D. A RCT for assessment of active human-centred learning finds teacher-centric non-human teaching of evolution optimal. NPJ Science of Learning, (in press) (2020).

Written by

Laurence Hurst is a Professor of Evolutionary Genetics, President of the Genetics Society, Director of the Milner Centre for Evolution and founder and director of the Genetics and Evolution teaching Project at the University of Bath, UK.

Momna Hejmadi is Professor of Bioscience Education and Technology and the Faculty of Science Associate Dean for Learning and Teaching at the University of Bath, UK.

Dana Buchan is an ex-school teacher (was head of biology at a local school) and recently was awarded her doctorate in evolution education research from the University of Bath.  She did all the hard work!

Laurence Hurst

Director of Milner Centre for Evolution, University of Bath

Evolutionary genomics

Comments

Go to the profile of Global ESD
10 months ago

I greatly appreciate the contribution from Prof. Hurst and colleagues here, even where I maintain some significant criticisms of this work (full disclosure, I served as an open peer reviewer on this paper for NPJ Science of Learning).

Hurst's concern that researchers could be missing interaction effects in more standard pair-wise testing designs is indeed critical, but the solution being offered does not really solve the problem (not that Hurst is claiming they are solving it fully).

As we argue in our recent article in Evolution Education Outreach (EEO) on the Educational Potential of Teaching Evolution as an Interdisciplinary Science (summarized on NPJ SciLearn here: https://go.nature.com/3qwX6o0) - there already exists a wealth of evidence that evolution science can and should be taught starting in the primary school. Here, I believe Hurst agrees with us that such primary grade evolution education is critical, and in the primary school context the "teach genetics first" mantra no longer holds. In this way, it is equally if not more likely that dominant interaction effects will occur at larger scales of curriculum design than within a unit on evolution. Hurst et al. may suggest that their intervention is more practical for teachers than wholesale curriculum reform, and yet if wholesale curriculum reform is what is needed, then optimizing lesson sequences or teaching practices at a lower level will not get us to where we would like to go in terms of evolution science literacy at scale. 

Against this background, Hurst's claim at the heading of this article is a generalization that does not hold water:


"Despite orthodoxy, active learning and human-centric subject matter do not make for the best way to teach evolution."

As we argue in our EEO article "evolution education is climbing the wrong mountain" by constraining the definition of evolution to an idealized gene-centric model grounded in 20th century thinking and not representative of 21st century advances in cultural evolution science (and other disciplines informed by evolutionary theory). Hurst's learning assessment tool is unabashedly grounded in this gene-centric view - and in this regard their "human-centric" treatment falls remarkably short from the kind of relevance and cognitive accessibility we describe as the educational potential of teaching evolution as an interdisciplinary science in the human domain.  

In sum, the work of Hurst et al. (2020) does call for an important push for empirical rigor in the evolution education research community, and for this it is more than worthy of publication, but it is not without fault. Empirical rigor without firm grounding in the underlying assumptions of the constructs to measured or the nature of curriculum change will only get us so far. Their grounding in an idealized gene-centric model of evolution necessarily constrains their exploration of the learning potential of human-centric approaches in evolution education. Additionally, their 'practical' focus on lesson sequencing and teaching approaches obscure the needed complexity our field must embrace to make the progress that students and teachers around the world need from our educational research community. 

The evolution education research community needs to get better at (A) grappling with the pedagogical implications of conceptual change in scientific perspectives of evolution in the human domain, and (B) advance a more open, networked, and interdisciplinary research community that has the social and conceptual capacity to provide systems thinking and research into whole curriculum design and school improvement efforts.

Until then - the Hurst et al. article, when contrasted against our own recent work, provides important fodder for productive disagreements about the research directions our field should be taking. 

-Dustin Eirdosh
Max Planck Institute for Evolutionary Anthrpology
Dept. of Comparative Cultural Psychology
Co-Founder www.GlobalESD.org