Information on how we evaluated LEANS in primary schools (2021), and our evaluation paper. New for September 2023: Read our evaluation study pre-print on OSF (free): https://osf.io/fhc2k/ Introducing the study While teachers and other education professionals were involved throughout development of the LEANS resources, there’s no substitute for trying out resources in real classrooms. The evaluation study had two main goals: To test LEANS as a neurodiversity teaching tool. Would children who participated be able to show knowledge of neurodiversity, and would their attitudes change? To test whether LEANS resources would be usable and practical in real classrooms. To meet those goals, we created two quizzes that tried to measure the specific ideas taught in LEANS and collected qualitative information from open-response quiz questions, written teacher diaries, and interviews with a subset of children. This evaluation was pre-registered. View pre-registration for the quantitative components of this study on the Open Science Framework (OSF) The following sections describe the study in a way intended to be accessible for teachers and families. It’s light on tables and statistical information! Who participated in the study? Four Scottish mainstream primary schools volunteered to test the LEANS resource pack between August-December 2021. Two schools were in smaller, more rural communities, and two were larger schools located in cities. Eight classes participated in total, with pupils aged 8-11 (years P5-P7, including some mixed-age classes). About 160 children were exposed to the LEANS resources through this whole-class delivery. One class chose to withdraw from the study partway through, meaning that 139 children finished the LEANS programme. Of these, 62 children were enrolled in the LEANS evaluation study. This means we had permission from parents/carers to analyse their quiz scores. Thank you 2021 was an extraordinarily difficult and demanding year for schools across Scotland, the UK, and the world. We offer our sincere and heartfelt thanks to our participating schools, who showed such commitment to LEANS in the face of so many other demands. We are equally thankful to all the schools who reached out and offered to support LEANS evaluations in other ways. LEANS measures The LEANS curriculum aims not only to teach new knowledge about neurodiversity, but to influence pupils’ actions and intentions toward greater inclusiveness, respect, and kindness (our know-think-do goals). We created two bespoke quizzes that tried to measure the specific ideas taught in LEANS, one focused on attitudes-actions and the other on neurodiversity knowledge. Quizzes and teacher quiz instructions Our quizzes are freely available for schools to use, if they follow the terms of use. The quizzes and scoring guide are linked from our study pre-registration on OSF. Please also download the Teacher quiz instructions, which you will need, but could not be added to the same file location on OSF (apologies!). Document LEANS teacher quiz instructions (897.63 KB / PDF) Evaluation results Multiple choice quiz scores showed that after LEANS, significantly more children selected courses of action that aligned with the accepting and inclusive principles in LEANS. Example actions agreement question: Image “In my class, most people are kind and friendly to people who are different than they are.” We predicted that the agreement question scores would also be higher after LEANS, but actually found a decrease across the group. We think that some children may have become more reflective or analytical about their class after taking part in LEANS. For example, per the example question above, children may have changed their minds about what “kind and friendly” actions look like, or about what their own classmates seem likely to do. We don’t know if this interpretation is correct, but we do believe it’s positive if children’s views evolve as they encounter new ideas about diversity. We asked a neurodiversity definition question both at the beginning and end of LEANS to estimate how many children knew this concept (box below). This was a multiple-choice question with 5 options. Before LEANS, just 17.7% of study participants chose the correct definition of neurodiversity, slightly worse than chance. After LEANS, 59.7% of participants were able to choose the correct definition, an increase of 42%! Neurodiversity definition question (before and after LEANS): What is the best explanation of the word neurodiversity? In a group of people, there will be differences in how they look, where they come from, and what they like or dislike. Every person’s brain has many different kinds of neurons in it, and they do different things. People experience the world differently because everyone’s brain is a unique shape and size. In a group of people, there will be differences in how their brains process information, and how they experience the world. I don’t know. When we asked children about the neurodiversity facts they had encountered in LEANS, we also found that the participants’ average score of 5.08 out of 8 significantly exceeded the chance level of 1.6 correct answers. As a group, children who participated in LEANS successfully demonstrated neurodiversity knowledge and a grasp of new vocabulary after the programme. Results from our child interviews suggest that children were also able to talk about these new ideas, not only choose them on paper. What did children and teachers think of LEANS? A thematic analysis of interviews about seven children's experiences of LEANS indicated that they found the experience fun, and it was covering new content. One explained, “I had heard of [neurodiversity] before I just didn't exactly know what it meant.” We found no evidence of upset amongst the interviewed children who had taken part. Critically, they considered it a ‘regular’ part of their school experience. LEANS did not stand out as unusual or separate from other topics they covered. This is very important, given issues about diversity and inclusion topics too-often being presented as separate, or concerning some people and not others. The interviews also showed convincing evidence that participating children had taken on board key messages about differences between people relating to neurodiversity and individual LEANS topics, and were changing their views. One child reflected that “if I treat everyone the same way it could be unfair” (LEANS encourages equity-based approaches to fairness). On communication, another “learned that not everything needs to be said with words.” Another participant summarised their LEANS learning by explaining that "everyone is different and that you don't have to be the same as everyone else... you can just be you”. In the teacher diaries, we received many valuable suggestions about simplifying and improving the resources. These have been fed into the final versions available now! Asked to reflect on the perceived impacts of LEANS in their classes, here are some thoughts they shared: When I spoke to parents of ‘neurodivergent’ children, they were very happy we were exploring this concept in class. It has been a great resource where pupils have enjoyed the activities. I think that studying LEANS genuinely made an impact on the class. Individual children felt and explained that they recognised themselves as neurodivergent (in different ways) and they appeared to have a ‘penny drop’ moment…. I think it helped children to have a better understanding of the way others are/feel, and also why learning can be different and is delivered differently to them. There are a lot of individual needs in the class and so I’m sure some of the stories will have resonated. Pupils were able to relate aspects of the project to children in the school. They realised that having a laptop at writing time [and other supports] all allowed children to have the same access to the curriculum. They also realised that… ‘tantrums’ could be because that pupil was struggling and needed some support [and] understanding. There is a better sense of empathy within the class, an understanding, a tolerance and acceptance. Overall, teacher comments suggested that there were some important connections and insights in their classes, but also more learning to do in the future. For example, several teachers pointed to examples of their pupils’ incomplete understanding of the content—like pupils understanding needs as important but still wanting their own needs to come before others, or “[empathising] with the characters in the stories, but [not always seeing] the behaviours in themselves.” This unevenness makes sense: children’s development and understanding will always vary across a class, and LEANS is one step in a longer path towards improving understanding and acceptance at school. Looking for more about the evaluation study? Read our evaluation study preprint on OSF (free). A preprint means this is the author version of the evaluation study journal paper. It is currently being peer-reviewed for publication. Alcorn, A. M., McGeown, S. P., Mandy, W., Aitken, D., & Fletcher-Watson, S. (2023, September 1). Learning About Neurodiversity at School (LEANS): Evaluation of the LEANS resource pack in mainstream primary schools. https://doi.org/10.31219/osf.io/fhc2k The quantitative part of the evaluation study was pre-registered, and you can find further details and full text of the LEANS quizzes on the Open Science Framework (free, no log-in required). View pre-registration for the quantitative components of this study on the Open Science Framework (OSF) We will share academic publications and public reports as they become available. Check our publications and reports page In 2022, we’ll also be presenting about the evaluation study at in-person and online conferences. Find out more on our 2022 conferences page This article was published on 2022-10-13
