Our research process

At Talamo, our mission to support individuals with Special Educational Needs (SEN) is powered by a commitment to creating a statistically valid, reliable, and accurate dyslexia testing and cognitive profiling tool. This tool is designed using best-in-class methods to be trusted by candidates, schools, and parents and we've followed a 5 stage process when creating and improving the test:

Our stages of research

Creating the test material

We start by collaborating with globally renowned publishers and psychometrics experts to develop our test content. This stage aims to replicate as much of the depth of a full diagnostic assessment as possible in a more concise format. Through this collaborative process, we crafted a diverse test battery that accurately measures areas such as verbal, visual, and non-verbal reasoning, phonological processing, working memory, processing speed, reading, and spelling. We purposely created more content than we’d need to choose the most effective content for each area.

User testing the material

This phase involves thorough user experience tests to assess each test's clarity, appropriateness, and difficulty. We split our user testing into stages to make improvements between each session. Feedback gathered during this phase informs the refinement or removal of test content ahead of more extensive data collection, ensuring that the foundation for our assessments is solid and user-friendly.

Evaluating each tests sensitivity to dyslexia

A pivotal element of our validation process is the direct comparison between dyslexic and non-dyslexic groups, using a sample of 200 students from each category. This stage involves partnerships with specialist dyslexia schools (to take formally diagnosed dyslexic pupils) and collaboration with SENCos to accurately define our ‘non-dyslexic’ control group. Focusing on students aged 12-13 helps mitigate age-related variability and provides a stable basis for our analysis.

Creating the model

After completing Stage 3, we had a large battery of tests which showed good statistical significance in differentiating between dyslexic and non-dyslexic. The next stage was to build a model which would use multiple tests to accurately spot dyslexia.

We experimented with multiple statistical methods but settled on a Logistic Regression as the most effective. With this model in place, we then used Recursive Feature Elimination to identify the tests which were most effective at spotting dyslexia (also taking into account how long the test is). We built multiple models based on this approach and then chose the final test battery based on the model's sensitivity and specificity score. Our current model now sits at 94% accurate.

As we build our data set, this number will only increase.

Age standardising

Evaluating sensitivity to dyslexia

Once we had our test battery locked in and the model in place, we then began age standardising to ensure our tool's applicability across different age groups. We collected extensive data for ages 7-16 to achieve statistically valid age standardisation. By collecting data across age groups, we can adjust the difficulty of our tests and interpret results based on age. In this phase, we derive standard scores, z-scores, and percentile ranges for each age group on every scale. This precision is vital for pinpointing underlying issues and offering relevant recommendations. This is where the bulk of our 1000 users were tested.

Age standardising

This crucial step ensures our tool's applicability across different age groups. We collected extensive data for ages 7-18 to achieve statistically valid age standardisation. By collecting data across age groups, we can adjust the difficulty of our tests and interpret results based on age. In this phase, we derive standard scores, z-scores, and percentile ranges for each age group on every scale. This precision is vital for pinpointing underlying issues and offering relevant recommendations. This is where the bulk of our 1000 users were tested.

Considerations

Care when collecting data

Our commitment to creating a diverse and representative dataset shines through our collaborative efforts with schools across the UK, ensuring optimal testing conditions and maximum engagement. We ensured a demographically accurate spread and attended each school numerous times to oversee the collection. See the schools we've partnered with.

What next?

Our dedication to refinement and innovation is ongoing. With each test, piece of feedback, and new piece of research, we continually enhance our tool's reliability and effectiveness.

At Talamo, we're committed to supporting neurodiversity through continuous innovation, research, and an in-depth understanding of the communities we serve.