Previous posts in this series considered the differences between process- and strategy-based training, and what success might look like in a training study. Another key design feature to think about is the inclusion of an adequate control group. Including a control group seems obvious, since we need to make sure that gains are linked to the training rather than to normal development. But designing a study with a good control is challenging.
A control group might be matched to the training group on key characteristics, such as general cognitive ability and age, only differing in that they do not receive the training – a ‘business as usual’ control. While this seems like a sensible option, it is important to consider what the control is doing while the other group receives training. Is the control group doing normal reading practice while the training group get their reading intervention? Or perhaps the training group is receiving extra reading help while the control group have already left school for the day. This is clearly an important distinction that will affect the conclusions that can be drawn from the results.
One way to counter these challenges is to include an active control group. In this scenario, the control group is again matched on key characteristics to the training group. However, the control group also receive some training, just not in the skill that is trying to be developed. In this case, the control group could be given something very different to the reading training group, like a maths intervention, both taking place after school so as not to interfere with normal schooling. This would mean that any gains seen in the reading training group cannot be down to the effect of simply taking part in a piece of research, which might involve working on fun computer programmes or with researchers and cause a spike in engagement at school.
But is this a fair comparison? Is it very surprising if pupils improve their reading skills after doing some more reading? If we really want to find out what causes the change, we need an even closer match for the control group – for example a similar reading intervention that does not train the key ingredient that is thought to lead to improvement (e.g. phonics). Now if we see an improvement in the training group, we can be fairly sure that there is something special about the phonics training that led to gains.
The use of different types of control group is associated with recruitment challenges that should also be taken into account. Unsurprisingly, many teachers and parents are opposed to their children being put into the control group, which can mean that fewer pupils sign up to take part. One clever way around this is to use a cross-over ‘wait list’ control group, where half of the pupils are in the training group and half are in the control group for one phase of the study, then they switch for the second phase. Everyone receives the training at some point, and it is still possible to compare training to control.
A final option is to include no control group. This might be appropriate when the aim is to see which individuals respond best to the training. For instance, do those with better working memory improve more with phonics training than those with poorer working memory? To answer this question, a group with a large variation in working memory skills could take part, with no control group. In this example, the outcome will be able to tell us something useful about the mechanisms of learning in the absence of a control group.
There is no single right answer when it comes to choosing what the control group does. This will vary between studies and should be thought about very carefully before commencing the study, depending on what the research question is.
This post first appeared on my personal webpage.