Using AI to help children get ahead in handwriting

While writing comes as second nature to most of us, it’s a complex process that takes time to develop. When children are learning to write, they combine fine motor skills (such as holding the pen or pencil), language, memory, and concentration. No simple task when you’re four!

School Rebound has developed an innovative app for digital tablets to support children as they learn to write. As well as fun games and exercises, the app provides a deeper analysis of handwriting issues using machine learning data. The technology allows children to practice specific skills and elements of writing to help them improve.

We talked to Thibault Asselborn about how the technology works.

Can you tell me more about the problem School Rebound is helping the solve?

Around 25% of children experience difficulties with learning how to write at some stage. For some children, the issues are mild and can be overcome in a few months. For some children, handwriting can be more than just difficult. Dysgraphia is a learning disability that impairs the ability to write. Children with dysgraphia can have problems spelling, forming letters, or even holding a pen.

But for children that have dysgraphia, their struggle with handwriting can have a huge effect their success at school, as well as the knock-on impact on their behaviour and self-esteem.

It’s crucial to identify issues with handwriting early on, so teachers, parents, and educational therapists can work with the children to successfully overcome their challenges.

Currently, handwriting issues such as dysgraphia are diagnosed after a child is referred to a therapist. The child completes a pen and paper writing test, and the therapist will analyse the static handwriting. They tend to us criteria such as how the child forms letters, legibility, and whether the writing is shaky or smooth. This method can be subjective, however. Studies show that there’s disparity in the diagnosis with only 60% of evaluators coming to the same conclusions on a test. Crucially, it doesn’t take into account the dynamics of handwriting. By this I mean things such as speed, or tilt and pressure of the pen. Our research has shown this is a critical element of understanding dysgraphia.

So how does your technology help?

Our solution – Dynamico - is for children between 4 and 12 years old. We’ve combined the capabilities of digital tablets and machine learning to support children (as well as their teachers and parents) as they start to learn to write.

The app is based on adaptive learning. Each child will have their own specific handwriting profile. This deep analysis identifies their handwriting strengths and weaknesses – how they hold the pen, the pressure they put on the paper, the speed they write. This diagnosis is quick and, most importantly, objective.

As well as the handwriting profile, the app will suggest specific areas the teacher can work on through games and exercises. We can all remember from school the boredom of having to do endless handwriting exercises! Our games are designed to be fun and engaging, rather than just copying letters. For example, one game specifically works on the child’s control of pressure, while other games work on control of speed or dexterity of the fingers.

How did you come up with the idea for School Rebound?

I was initially researching the role of social robots in teaching handwriting to children, and I became more interested in the mechanical aspects of handwriting. School Rebound and Dynamico (our app) evolved from there. We’ve been working closely with our three core markets - schools, parents, and educational therapists - to develop the application to ensure it meets their needs.

The app has a subscription model, parents will be able to buy a monthly subscription to help diagnose and remediate handwriting issues at home, and educational therapists and schools will be able to purchase an annual subscription.

Can the technology be used in other languages?

We’ve developed the app in French, but our model is universal. The machine learning looks at data at a much deeper level than just looking at the letter on the page. We’ve found no matter what language is being written in, the same issues present. A child who has difficulty controlling the pressure of the pen when they write in Italian will have the same challenge if they were to write in English.

Furthermore, we’ve also found the model can be used with the Latin and Cyrillic alphabets, and we believe it can be transferred to other languages as well.

How will you use the Innogrant?

I’m pleased to say we’ve already launched Dynamico in the Swiss app store. By the 1^st January 2022, we’re aiming for the app to be available in France, Germany, Italy, and the UK – expanding our potential market to over 24m children across Europe.

Are there any other applications that your technology could be used for in the future?

Our ambition is to use our artificial intelligence approach to help better identify other children’s learning difficulties, such as dyslexia.

There are also several potential medical applications. We would like to explore whether the technology could be adapted for identifying carpel tunnel syndrome or used to detect conditions such as autism and Parkinson’s.

School Rebound is based in EPFL’s CHILI Laboratory (Computer-Human Interaction in Learning and Instruction), supported by Professor Pierre Dillenbourg.