Introducing the study
While teachers and other education professionals were involved throughout development of the LEANS resources, there’s no substitute for trying out resources in real classrooms.
The evaluation study had two main goals:
- To test LEANS as a neurodiversity teaching tool. Would children who participated be able to show knowledge of neurodiversity, and would their attitudes change?
- To test whether LEANS resources would be usable and practical in real classrooms.
To meet those goals, we created two quizzes that tried to measure the specific ideas taught in LEANS and collected qualitative information from open-response quiz questions, written teacher diaries, and interviews with a subset of children. This evaluation was pre-registered.
The following sections describe the study in a way intended to be accessible for teachers and families. It’s light on tables and statistical information!
Who participated in the study?
Four Scottish mainstream primary schools volunteered to test the LEANS resource pack between August-December 2021. Two schools were in smaller, more rural communities, and two were larger schools located in cities. Eight classes participated in total, with pupils aged 8-11 (years P5-P7, including some mixed-age classes).
About 160 children were exposed to the LEANS resources through this whole-class delivery. One class chose to withdraw from the study partway through, meaning that 139 children finished the LEANS programme. Of these, 62 children were enrolled in the LEANS evaluation study. This means we had permission from parents/carers to analyse their quiz scores.
Thank you
2021 was an extraordinarily difficult and demanding year for schools across Scotland, the UK, and the world. We offer our sincere and heartfelt thanks to our participating schools, who showed such commitment to LEANS in the face of so many other demands. We are equally thankful to all the schools who reached out and offered to support LEANS evaluations in other ways.
Evaluation results
The LEANS curriculum aims not only to teach new knowledge about neurodiversity, but to influence pupils’ actions and intentions toward greater inclusiveness, respect, and kindness (our know-think-do goals). We created two bespoke quizzes that tried to measure the specific ideas taught in LEANS, one focused on attitudes-actions and the other on neurodiversity knowledge.
Multiple choice quiz scores showed that after LEANS, significantly more children selected courses of action that aligned with the accepting and inclusive principles in LEANS.
Example actions agreement question:
“In my class, most people are kind and friendly to people who are different than they are.”
We predicted that the agreement question scores would also be higher after LEANS, but actually found a decrease across the group. We think that some children may have become more reflective or analytical about their class after taking part in LEANS. For example, per the example question above, children may have changed their minds about what “kind and friendly” actions look like, or about what their own classmates seem likely to do. We don’t know if this interpretation is correct, but we do believe it’s positive if children’s views evolve as they encounter new ideas about diversity.
We asked a neurodiversity definition question both at the beginning and end of LEANS to estimate how many children knew this concept (box below). This was a multiple-choice question with 5 options. Before LEANS, just 17.7% of study participants chose the correct definition of neurodiversity, slightly worse than chance. After LEANS, 59.7% of participants were able to choose the correct definition, an increase of 42%!
Neurodiversity definition question (before and after LEANS):
What is the best explanation of the word neurodiversity?
-
In a group of people, there will be differences in how they look, where they come from, and what they like or dislike.
-
Every person’s brain has many different kinds of neurons in it, and they do different things.
-
People experience the world differently because everyone’s brain is a unique shape and size.
-
In a group of people, there will be differences in how their brains process information, and how they experience the world.
-
I don’t know.
When we asked children about the neurodiversity facts they had encountered in LEANS, we also found that the participants’ average score of 5.08 out of 8 significantly exceeded the chance level of 1.6 correct answers. As a group, children who participated in LEANS successfully demonstrated neurodiversity knowledge and a grasp of new vocabulary after the programme. Results from our child interviews suggest that children were also able to talk about these new ideas, not only choose them on paper.
What did children and teachers think of LEANS?
A thematic analysis of interviews about seven children's experiences of LEANS indicated that they found the experience fun, and it was covering new content. One explained, “I had heard of [neurodiversity] before I just didn't exactly know what it meant.” We found no evidence of upset amongst the interviewed children who had taken part. Critically, they considered it a ‘regular’ part of their school experience. LEANS did not stand out as unusual or separate from other topics they covered. This is very important, given issues about diversity and inclusion topics too-often being presented as separate, or concerning some people and not others.
The interviews also showed convincing evidence that participating children had taken on board key messages about differences between people relating to neurodiversity and individual LEANS topics, and were changing their views. One child reflected that “if I treat everyone the same way it could be unfair” (LEANS encourages equity-based approaches to fairness). On communication, another “learned that not everything needs to be said with words.” Another participant summarised their LEANS learning by explaining that "everyone is different and that you don't have to be the same as everyone else... you can just be you”.
In the teacher diaries, we received many valuable suggestions about simplifying and improving the resources. These have been fed into the final versions available now!
Asked to reflect on the perceived impacts of LEANS in their classes, here are some thoughts they shared: