Many Fingers in Many Pies.

During the tests, Zed Text was directly compared with – and outperformed – Helvetica in terms of speed of reading for all types of hospital patients. We have since further improved the speed and ease of reading by creating wider versions of Zed that benefit readers with declining vision.

And this is interesting too

So yesterday Michael in our team chatted through a client

Deteriorating Neurological condition. Signifcant VI - requires huge font Black on White background. Difficulty with Fine motor control but is literate. Tends to mouth words but not easily undersood.

She needs the letters so large we could use echo but we have to put it under apples magnifier - its not easy to setup.

Hmmmm. He is right. When we made pasco - you could do this. It allowed you to make the font so large it could show one letter at a time. But its not ideal (and that was super buggy). And Echo - we cant do that with. I need to fix it.. But..

Me: Do you need prediction?

Michael: Hmm yes - possibly

Me: Do you want next letter prediction:

Michael: Ideally.. yes

Hmmm. So what if we tear up the rule book (RICE scores does trigger this quite highly due to the rapidly deteriorating solution but really I dont like re-creating something..) and make something. Over the last 12 months I’ve made a ton of different stuff. All of these things were for a particular project but I’ve been careful to turn a lot of into libraries or standaline code. These “bricks” I always thought would be useful.. a JS word prediction system, a WorldAlphabets library and a TTS Wrapper in Node and Python…So we have the building blocks. What if I got a LLM to make this.. its pretty simple..

15 Minutes of prompting with Gemini (and then a bit of hacking - which took 4 times as long from me) I give you…

willwade.github.io/letter-by…

Code at : github.com/willwade/…

.embed-container { position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; } .embed-container iframe, .embed-container object, .embed-container embed { position: absolute; top: 0; left: 0; width: 100%; height: 100%; }

Its a React/Vite app (Gemini only does react but Im cool with that). And its leveraging our WorldAlphabets project and PPMPredictor. Note how it learns your own language and will improve over time.

Last week we were honoured to win the Samantha Hunnisett Access Award at the CM Awards.

To be honest, I wasn’t expecting it. Most of my attention was on the big-ticket award of the evening — the Lifetime Achievement Award for Alli Gaskin. Alli was an awesome SLT colleague and friend, and she will be deeply missed. (If I did have any clue we were up for it I might have prepared a coherent speech and not drank two bottles of wine before hand. Apologies to all those who heard my garble on)

But fittingly, Alli also had a direct connection to Echo. While working at Lancasterian she supported a child whose communication needs weren’t being met by any existing AAC system. Echo bridged that gap. She immediately recognised its value and became one of its strongest champions. That connection made winning the award all the more meaningful.

Where It Started

Echo itself began initally as “Pasco”. Named by Euan Robertson (Phrase Auditory Scanning Communicator) we had it drafted for a while as we had around 6 or 7 end users who simply weren’t being supported by the auditory scanning options available. But then one of our clients Paul Pickford one day reached out to us. Paul said:

“I’m selling some stuff and I want to donate some money.”

Paul’s donation gave us the chance to act. Using our RICE framework, we prioritised building this tool — knowing it could have a meaningful impact.

The first version was hacked together quickly, but it worked. People like Darren, recovering from a brain stem stroke, were among the first to try it. From the start, Pasco (and then Echo) was shaped directly by the people who needed it.

A Chinese ICU Case

One case that has always stayed with me came from the early days, when Echo was still called Pasco and lived as a web app.

I met a woman in intensive care with significant visual impairment and minimal movement. But the bigger barrier? She was a native Chinese speaker, surrounded by English-speaking staff. Translation tools let staff tell her what was happening, but she had no way to respond.

We added multilingual support into Pasco: staff could cue in English, but she could output in Chinese. For the first time, she could communicate back. Her parents — also Chinese-only speakers — were in tears. I tried explaining with Google Translate, but it butchered my speech so badly I gave up and just demoed it. Seeing her finally communicate was enough to make me realise it was worth it.

Pasco Becomes Echo

At that stage, Pasco was still rough around the edges. The two-switch scanning had bugs that were difficult to fix. That’s when Gavin Henderson rewrote it in SwiftUI. (Side note: SwiftUI is a dream. I love it.)

That rewrite became Echo.

And then something amazing happened. I was at a conference in Croatia, getting ready to speak, when I noticed a talk being given in another language. I couldn’t follow the words at first, but eventually I realised what was being demonstrated on screen: Pasco. Somehow, it had been picked up and shared across borders, because we had made it open and multilingual. That moment drove home how far-reaching these tools can become when you don’t lock them away.

Open Source and Ongoing Challenges

Echo has always been about more than one app. It’s about showing what’s possible — and letting others build on it. That’s why we’ve kept it open source: 👉 github.com/acecentre…

We’ve also learned when to change direction. For example, we once had a clever “auditory splitting” feature — playing different parts of the audio to different devices. It worked, but it was fiddly. And ultimately, Apple’s iOS audio system won’t let us split audio streams the way we need. Until that changes, it’s not practical.

But the point remains: innovation often starts with finding a solution for one person. When shared openly, those solutions ripple outward and benefit many.

A Team Effort

So yes, the Access Award has Echo’s name on it. But really, it belongs to: • Paul Pickford — whose donation started it all. • Hossein Zoda — who built the first version (Pasco). • Gavin Henderson — who rewrote it in SwiftUI and brought Echo to life. • Michael Ritson (Ace Centre) — for deep input on visual impairment and access aspects. • Charlie Danger — who once needed it working with a scroll wheel as input. • And most of all, the end users — Darren, the client in ICU, and many others whose needs shaped the app.

It’s also for the wider Ace Centre team, who foster and curate ideas like this every day by working closely with individuals with disabilities.

Cooking Up an AAC Correction Tool: A Recipe for Readability

Alright, gather ‘round, aspiring communication chefs! We’re about to embark on a culinary adventure – one where we cook up something genuinely useful for the world of Augmentative and Alternative Communication (AAC). Ahem - I just LOVE an analogy.. I may have taken this to far.. but just go with it for now.. All code is at https://github.com/AceCentre/Correct-A-Sentence/tree/main/helper-scripts - I wrote this actually some time ago. Some of the data maybe a little but outdated but I figured it maybe of interest to someone out there

Now, before we don our aprons, a little confession: I flunked computer science at uni. Like, properly failed the module. So, if I can bumble my way through making a “Fix Tool” from scratch (and hey, big shout-out to the Grid3 “Fix Tool”for showing us what’s possible – long may it reign!), then you absolutely can too. Consider me your fellow learner, not some all-knowing NLP wizard. Let’s learn by doing, shall we?

The Core Ingredient: Why Are We Even Doing This?

Think of it like this: AAC systems - They’re amazing, but sometimes, they’re not quite as zippy as we’d like. We need these systems to be faster and better quality output. This isn’t just my hunch; it’s what we hear from from our staff, clients, and years of digging into project feedback. We need efficiency.

What does that even mean in a world of AAC? Well, it could be speedier services, better quality support, or even device improvements like smarter input detection. But for today’s special, we’re focusing on the juiciest bit: Output. We want to churn out text – or “utterances,” if you’re feeling fancy – as quickly and clearly as possible.

The snag? Sometimes, the input data is, shall we say, a little “noisy.” This “noise” comes from typing fast, or when folks face access challenges. So, how do we clean up that messy input and support someone? My secret ingredient, the one I’ve been shouting from the rooftops for ages, is Auto-correction. A sidenote: I don’t mean auto-prediction here - I mean correction. Typing on those small keys on your phone. Its done by working out what you could have meant to say and nattily correcting it before you even see the text go into the writing area. When you see it enter wrong you think its bad - but when it doesnt you assume its your excellent writing skils.. Truth is its happening all the time.. But why isn’t it standard in AAC where mistakes are equally high?!

Step 1: Let’s Bake Our Own! (Seriously, it’s fun!)

Forget those fancy, pre-made mixes you just microwave. We’re going old-school, from scratch! (Seriously, don’t just ask an LLM to whip this up. You learn SO much by getting your hands dirty).

If you were to create your own AAC correction tool, you’ve got options. But first, let’s peek into our cupboard and see what kind of “noisy” sentences we’re often dealing with. These are our raw ingredients:

Ihaveaapaininmmydneck (No spaces AND typos – a double whammy!)

I wanaat a cuupoof te pls (Double presses – sticky keys, anyone?)

u want a oakey of cjeese please (Positional errors – hitting nearby keys)

can u brus my air (Deletions – vanishing letters!)

Can you help me? Can you help me? (Repeats – when a stored message goes wild!)

Generally, these sentences are short and sweet. Sometimes, folks use word prediction, and boom! Perfect sentence. But we all know word prediction can be a bit of a brain drain, right? All that visual scanning and mental parsing? Not always the most helpful, generally speaking (No seriously now - there is good research that prediction hasnt been that helpful - but we DO lnow of people who use it it a lot.. one key thing is that if you do want prediction keep the visual load by keeping it to three cells[^1])

The Toughest Nut to Crack: Writingwitjoutspacesandtypos

This first one is the making a meringue of our baking challenge. How do you find where one word ends and another begins when they’re all smushed together?

Enter Python and Grant Jenks’ awesome trie-based wordsegment library. This clever little tool uses “unigrams” and “bigrams” (think of them as single words and two-word phrases, each with a special “weight” based on how common they are) to figure things out. Here’s a taste:

from wordsegment import load, segment load() segment(‘thisisatest’)

# Output: [‘this’, ‘is’, ‘a’, ‘test’]

Pretty neat, right? But here’s the rub: language is super personal. Your favourite biscuit might be a “biccie,” but if “biccie” isn’t in wordsegment’s dictionary, it’s going to struggle. For example, Iwouldlikeabicciewithmycuppatea might come out as ['i', 'would', 'like', 'a', 'bi', 'ccie', 'with', 'my', 'cuppa', 'tea'].

You can still kinda figure it out, right? And here’s a crucial baking tip: SOMETIMES, “GOOD ENOUGH” IS PERFECTLY FINE! Remember most people arent commuicating to nobody - they generally have a communication partner and co-construction is everything in AAC. We don’t often need to do it ALL. A little help is often all that’s needed. But what if we want to perfect our pastry?

Adding Our Secret Family Ingredients (Personalization!)

What if we knew a person’s actual vocabulary? (Big “if,” I know!) We could sprinkle their unique words and phrases into wordsegment’s data. Imagine taking an end-user’s real-life text and using it to train the tool. It’d be like using your grandma’s secret spice blend!

Check out the docs on customizing the corpus and our own code here and here. Fun fact: we even made typo-heavy bigrams based on real typos! This is a solid start, though capturing every personal nuance is a challenge.

We could also try a “fuzzy” approach (like this blog suggests). It’s light on memory and doesn’t need a super-fancy GPU, which is great for on-device use. It might not be lightning-fast, but perhaps “quick enough.” Still, throw in acronyms, names, and weird abbreviations, and you’ve got yourself a truly tough cooking challenge!

The Fancy Pre-Made Sauce: Large Language Models (LLMs)

So, what about the new kid on the block, the Large Language Model? Let’s try wrapping our noisy sentences in an OpenAI GPT-3.5 Turbo bow and see what happens:

https://www.youtube.com/watch?v=ExCukoFpaWY&list=PLWWQ5nlUD_ttgsAsjaUAyJz7GfFqOCePR&index=1

Not bad, right?! You can instantly see the magic: focus on typing, less visual distraction from predictions, maybe even lower dwell times (0.5 seconds in that video is practically unheard of!). This could be a game-changer!

But here’s the colossal snag, the bitter aftertaste: All that super-personal data? It’s flying off to the cloud, potentially helping train a commercial entity’s models. And when we’re talking about someone’s unique communication, “personal” means really personal. So, while the pre-made sauce is convenient, it just doesn’t sit right for privacy. We need something that’s:

Efficient (Quick to cook!)

Privacy-focused (All in your kitchen, no external diners!)

AAC-Like text savvy (Understands our unique culinary style!)

The Holy Grail: Baking Our Own Language Model! (Woohoo!)

Okay, deep breaths. This sounds scarier than it is, I promise! To bake our very own language model, we need some fundamental ingredients:

Data. A whole heap of raw, noisy strings that looks like your real user input, plus the perfectly “cleaned” version it should become. The closer to your real sentences your users actually write, the better.

More Data.

Even More Data.

No, seriously, a LOT more Data.

Yep. Keep on going. More please.

A little bit of code (the “recipe card”) to “train” our system. Let’s not get bogged down in the fancy whisking techniques just yet. Lets just say you need a good food mixer!

Time, a bit of cash for the oven (aka computer somewhere), and a tiny pang of guilt about the electricity bill (/burning of the planet)

You get the picture: DATA is our main ingredient. But here’s the kicker: there’s barely any “AAC user dialogue” data out there. What is “AAC-Like text,” anyway?! Honestly, it’s a bit of a shocker that our industry hasn’t dug deeper into how people actually use AAC. Do they really use stored phrases all the time? What kind of language do they create? We’re mostly just guessing! (Im being purposefully harsh here - there is some great studies but genuinely not large datasets).

So, we have to “imagine” what AAC-like data looks like. Keith Vertanen made a fantastic attempt with his artificial dataset – kudos to him! We feel “spoken” corpora might be a good starting point, since AAC is essentially about “speaking” through a device. Let’s check out these corpora:

BNC 2014 Spoken

AAC Text

Daily Dialog

The catch? These are mostly perfectly transcribed texts. Not many glorious typos, which is what our grammar correction tool needs to learn from. We need truly “noisy” input. So how are we going to do that?

Injecting Chaos: Making Our Data Deliberately Messy

So, how do we take these pristine datasets and get them messy? We could artificially create the noise using a tool like nlpaug, designed for exactly this. But how “real” are these artificial typos? We don’t really know.

Ideally, we’d have real typos from the wild! Like those found here, or the TOEFL Spell dataset (essays by English language learners – goldmine!). And what about homonyms for those tricky folks who spell words like “they’re,” “their,” and “there” interchangeably? This list could be a good starter for ten. (Quick disclaimer: Not everyone does this, but it does happen, see what I just did there?).

So, for our Model Cooking Session, we now have:

A jug of spoken sentences – some homemade, some store-bought (! Im getting too carried away with this analogy right? I mean real utterances and some augmented like AAC Text)

A handful of real spelling errors.

And a good well made grinder to inject those errors right into our mixture!

We’ll take our pristine corpora and programmatically make them gloriously noisy by swapping in words from our typo datasets. Yum! (aka you need to write some code to actually do this.. Sadly this is where it isnt really like cooking. You do need to write some code..)

The Grammar Garnish: Adding Finesse

Now, what about grammar? I’m going out on a limb here, but a lot of AAC written text tends to have “poor” grammar. (Quick note: It’s not really “poor” – what matters is communicating! But technically, better grammar can help our text-to-speech systems sound more natural and grammar can fix what looks like misunderstandle sentences). Well like earlier we could take our beautiful ingredients (our copora) and strip out things like commas and apostrophes (which AAC users rarely include). But we could do with some better “base” material:

JFLEG Data: A respected source for grammar models. Sentences aren’t very “AAC-like,” but it’s a good foundation.

C4-200M: A staggering 200 million grammatically wrong sentences, corrected! It’s massive, so a smaller subset like this one is more manageable.

Our Grand Baking Plan:

So, the recipe is coming together! Here’s how we’ll prepare our training data:

Take each spoken text corpus.

Inject typos into the training data and strip out grammar elements.

Create three versions of each sentence:

With typos.

With typos and compressed (no spaces).

As it should be, but still with no spaces.

Also, add a grammar baseline training layer.

Then, we’ll bake it all using a clever “text-to-text transformer” model. We need something pretty lean, remember, because it needs to run efficiently on a device!

Time to Fire Up the Oven!

Want to try this yourself? Grab the script for preparing the data here (you’ll need to download the BNC2014 corpus and place it correctly).

It’s a lengthy process, full of detailed steps for adding typos, cleaning, and prepping the text. Remember that old chef’s adage: A good model is only as good as its ingredients! Here’s a snapshot of our data pantry (NB: This is a bit out of date - but you get the idea):

Data Source Average Word Length/Sentence Sentence Count Notes AAC Text 7.72 1,504 Artificially typo’d, 1,506 compressed variants BNC & Daily Dialog 9.71 885,186 Real typo’d, 889,546 compressed variants JFLEG Data 16.70 754 C4 Subset Data 9.76 41,394 Overall 9.4 1,914,226 File size: 196.78 MB, Split: 80/20 Train/Eval

Training itself is surprisingly straightforward if you have a decent GPU. Our batch took about 3-6 hours to cook (evaluation alone took two hours!). The delicious result? A model you can find here on Hugging Face.

The Taste Test: How Did Our Bake Perform?

Let’s see how our different “correction techniques” stack up on 39 test sentences (compressed and with typos):

The “algorithimic” approach Wordsegment + Spelling Engine: Around 14 seconds. Not bad for a quick fix! BUT it wont deal with anything that ISNT in its dictionary. So we can get this working in a test envrionment but we know it will fail with anything novel

Online Model: Azure/OpenAI GPT Turbo 16K: An astonishing 13 seconds! This is mind-blowing. It’s an HTTP call all the way to the cloud and back, yet it’s that fast. How?!

Ok so now, for our homemade models (This is different versions of our “cake” that I made):

Method Accuracy (%) Total Time (seconds) Average Similarity Inbuilt 0.0 17.32 0.93 GPT 55.56 13.29 0.92 Happy 28.95 N/A N/A Happy Base 13.16 N/A N/A Happy T5 Small 0.0 N/A N/A Happy C4 Small 0.0 46.90 0.81 Happy Will Small 28.95 N/A N/A HappyWill N/A 24.67 0.93

Here’s a little nibble of the results:

Incorrect Sentence Correct Sentence Output-Inbuilt Output-GPT Output-Happy Output-HappyBase Output-HappyT5 Output-HappyC4Small Output-HappyWill Feelingburntoutaftettodayhelp! Feeling burnt out after today, help! feeling burnt out aft et today help Feeling burnt out after today, help! Feeling burnt out today help! Feelingburntoutaftettodayhelp! Feelingburntoutaftettodayhelp! Feelingburntoutaftettoday help!! Feeling burnt out today help! Guesswhosingleagain! Guess who's single again! guess who single again Guess who's single again! Guess who single again! Guesswhosingle again! Grammatik: Guesswhosingleagain! Guesswhosingleagain!! Guess who single again! Youwontyoubelievewhatjusthappened! You won't you believe what just happened! you wont you believe what just happened You won't believe what just happened! You want you believe what just happened! You wouldn'tbelieve what just happened! Youwontyoubelievewhatjusthappened! Youwontyoubelievewhatjust happened!! You want you believe what just happened! Moviemarathonatmyplacethisweekend? Movie marathon at my place this weekend? movie marathon at my place this weekend Movie marathon at my place this weekend? Movie Marathon at my place this weekend? Movie marathon at my place this weekend? grammar Moviemarathonatmyplacethisweekend? Moviemarathonatmyplacethis weekend? Movie Marathon at my place this weekend? Needstudymotivationanyideas? Need study motivation, any ideas? need study motivation any ideas Need study motivation. Any ideas? Need study motivation any ideas! Need study motivationanyideas? Needstudymotivationanyideas? Needstudymotivationanyideas? Need study motivation any ideas! Sostressedaboutthispresentation! So stressed about this presentation! so stressed about this presentation So stressed about this presentation! So stressed about this presentation! So stressed about this presentation! Sostressedaboutthispresentation! Sostressedaboutthispresentation!! So stressed about this presentation! Finallyfinishedthatbookyourecommended! Finally finished that book you recommended! finally finished that book you recommended Finally finished that book you recommended! Finally finished that book you're recommended! Finally finished that book yourecommended! Finalfinishedthatbookyourecommended! Finally finished that bookyourecommended!! Finally finished that book you're recommended!

So, yeah! Pretty sweet, right? We’re actually outperforming GPT – the fancy, paid online service – with our own home-baked model! That speed, though… and let’s not even mention the memory usage differences.

But here’s something to chew on: the “inbuilt” non-LLM technique (that Wordsegment one). It’s surprisingly readable, and no memory issues to worry about!

The Next Dish: A Reality Check

Now, for a live demo of our creation (it’s like showing off your perfect soufflé!):

Watching this, you’ll instantly spot a slight issue with our “test kitchen” data. It’s actually not as messy or noisy as our real-world, imagined scenarios. Most test sentences only had a couple of typos, whereas our real AAC input can look like every word has had a wrestling match with the keyboard.

So, the next step in our culinary journey? We need to hunt down, or create, an even richer, noisier, and more authentic corpus of data. The quest for the perfect ingredients continues!