Home > Solutions > Zaimplantowani – Speech Perception Module (MPM)

Zaimplantowani

A new rehabilitation module helping cochlear implant patients practice speech understanding in everyday life

About the project

The Faculty of Physics at the University of Białystok runs ZAIMPLANTOWANI (EARPLANTED) – a platform supporting the rehabilitation of cochlear implant users. Until now, the platform helped patients practice melody recognition. Our task was to add an equally important second pillar of rehabilitation: speech understanding practice.

The result is the Speech Perception Module (MPM) – patients choose any material that interests them (an audiobook, their own voice recording, or even a plain e-book that the system automatically converts into speech), listen to it fragment by fragment, and repeat the words they heard out loud. The application automatically checks how accurately the patient understood the text and shows this in a simple, numerical way – so patients can see their progress week over week.

Project goal

Hearing rehabilitation after cochlear implantation is a long, demanding process – often tedious and discouraging. The goal of the module was to make daily exercises as effortless as possible, while giving both the patient and the doctor a clear picture of progress.

The module was designed to provide:

  • practice on material the patient actually chooses and enjoys (a favourite book, their own recording) instead of artificial, imposed exercises,
  • automatic checking of how well the patient understands what they hear, without requiring a therapist’s involvement on a daily basis,
  • two difficulty levels – for patients starting rehabilitation and for those already doing well who want to focus only on the more challenging parts,
  • availability in five languages (Polish, English, German, French, Italian),
  • ease of use for people with disabilities – large, clear buttons and a minimal number of clicks to start an exercise,
  • full compliance with GDPR, since the module processes sensitive data, including recordings of the patient’s voice.
Project challenges

The biggest challenge was accounting for the fact that no automatic speech recognition system is 100% accurate – it can occasionally mishear something, just like a human would. We had to design a way for patients to correct such system mistakes without risking damage to the history of their previous exercises.

Another challenge was that the university’s server does not have a dedicated graphics card to accelerate AI processing. The entire speech recognition and generation pipeline had to be designed to run efficiently on a standard processor – while remaining easy to scale up should the university later invest in more powerful hardware.

A separate challenge was scoring patient results fairly – making sure that a single short, poorly understood fragment doesn’t skew the statistics as much as a long fragment full of errors, and vice versa. This required a carefully thought-out way of calculating results across an entire practice session, rather than just a single utterance.

On top of that, the module had to plug into the existing platform, which has been running for years, in a way that lets patients log in from one place without noticing they are actually using two connected systems – and the deployment itself couldn’t take the platform offline for more than a few days.

Our approach

The starting point was a detailed specification from the university’s team, describing step by step what the exercise should look like from the patient’s perspective – from uploading material, through listening and repeating, to reviewing the result. Based on this, we designed the entire application and the way patient data is stored securely.

We made sure that logging into the new module is invisible to the patient – they enter the existing platform, click a single tile, and land directly in the exercise, without having to enter their password again.

Alongside building the application, we prepared a script for an instructional video that walks patients through using the module step by step – from registration, through the first exercise, to reviewing their results. Like the module itself, the video is available in five language versions.

Architecture and technology

We built the application on a modern web stack (Next.js, NestJS, MariaDB database), containerized with Docker, which makes it easier for the university’s team to maintain and further develop.

For speech recognition and text-to-speech conversion, we used proven, open-source AI engines (WhisperX and Piper TTS) running locally on the university’s server – without sending patients’ voice recordings to external, paid cloud services. This matters not only for cost reasons, but above all for privacy – sensitive patient data never leaves the University’s infrastructure.

Processing of longer recordings (e.g. entire audiobooks) happens in the background, so patients don’t have to wait in front of the screen – they can close the application and come back once the material is ready.

Performance and security

All communication within the module is encrypted, and access to each patient’s data is strictly separated – no one can see another person’s materials or results.

The system meets GDPR requirements: a patient can request deletion of their account and data at any time, and materials that are no longer in use are automatically removed. The original files uploaded by the patient (e.g. an audiobook fragment) are deleted from the system as soon as they’ve been processed, which additionally protects against copyright infringement.

Results
  • A new rehabilitation module running under one, familiar patient account on the ZAIMPLANTOWANI platform.
  • Speech recognition and synthesis running locally on the university's server, with no external AI costs and no patient data sent outside.
  • A clear, numerical indicator of progress in speech understanding, meaningful to both the patient and the specialist guiding the rehabilitation.
  • An interface and instructional video available in five languages, designed with people with disabilities in mind.
  • Full GDPR compliance in handling sensitive patient data, with the module ready for further expansion in the future.