News

AGH University and Proassist to create voice assistant for doctors

Kolorowa grafika dekoracyjna. Po lewej stronie część profilu twarzy ludzkiej. Z ust wydobywa się dźwięk zobrazowany wykresem fali dźwiękowej.

Image source: Dreamstime

AGH University and Proassist to create voice assistant for doctors
research and science

​​​​​​​As part of the competition organised by the National Centre for Research and Development INFOSTRATEG, commissioned by the Ministry of Health, the AGH University and the Proassist company will create an intelligent voice assistant for doctors and medical staff. For this purpose, the Consortium received a grant of PLN 9 million.

The project involves the creation of a voice assistant that recognises the user's natural speech and performs the assigned tasks supporting doctors and medical staff in their day-to-day duties. With this solution, doctors will be able to give voice commands, which will facilitate, among other things, the issuing of e-referrals, e-prescriptions or sick leaves. The overarching goal of the project is to optimise doctors' working time and reduce the documentation work related to patient service. Such a solution will translate into increasing the specialist's attention during the visit solely on the patient and reducing the duration of the appointment and consultation itself.

Carried out by Proassist and a team of researchers from the AGH University of Krakow, the voice assistant project for doctors involves collecting voice material in the first stage. In this respect, doctors will record their voice during visits and consultations with patients. The collected data will be used to build a database and improve the voice assistant. Real-life data from medical appointments is the project’s key element, enabling the voice assistant to be tailored to the specific needs and preferences of doctors. The knowledge and experience of medical professionals accumulated over years will provide valuable input into the development of the solution.

Responsible for the implementation of the project on the part of the AGH University is Dr Konrad Kowalczyk, associate professor at the AGH University, from the Institute of Electronics at the Faculty of Computer Science, Electronics, and Telecommunications, acting as R&D manager in the project, who explains:

"Our main goal is to facilitate and improve doctors’ work by converting their speech to text and then automatically extracting desired information and completing the electronic medical record forms accordingly. We are working on the development of a voice assistant adapted to the medical use which will consist of artificial intelligence modules responsible for automatic speech recognition and natural language processing."

Prof. Konrad Kowalczyk's team is made up of about 10 people: assistant professors, assistants, doctoral studies students, and students connected with the AGH Signal Processing Group of the Faculty of Computer Science, Electronics, and Telecommunications, which has specialised in speech processing and analysis for many years now. In the initial phase of the project, the team will be responsible, i.a. for the preparation of a system enabling the conversion of speech into text along with semantic analysis of utterances, and in further stages of the project, for the creation of a generic voice system containing automatic speech recognition and natural language processing modules supporting defined usage scenarios. The research and development work will be led by experienced assistant professors at the Faculty of Computer Science, Electronics, and Telecommunications, Dr Stanisław Kacprzak and Dr Marcin Witkowski.

Photo from the AGH University archives

Zdjęcie grupowe zespołu naukowców z AGH wykonane na schodach w holu budynku głównego uczelni.

Jacek Piaseczynski, Proassist's CEO and the project manager on Proassist's side, emphasises:

"We want for doctors to communicate with their office computer or telephone as if they were talking to their medical assistants. We have an ambitious task ahead of us – we want to revolutionise the Polish health service and disseminate new technologies in medicine so that they can be used by every doctor in their practice. Expert support from the AGH University of Krakow, particularly in the areas of speech and sound signal processing or machine learning, certainly constitutes to our project team being even better prepared for this task."

The designed voice assistant for doctors will be equipped with advanced features that allow doctors to give instructions, including prescribing or creating treatment plans. This will enable doctors to optimise their time per patient and focus even more on direct patient contact, thereby improving the quality of medical care.

The project will involve medical consultants and medical facilities who will record conversations with patients. Subsequently, the conversations will undergo a process of, i.a. anonymisation, i.e. they will be transformed in a way that makes it impossible to attribute individual information to a specific or identifiable natural person and will thus be deprived of sensitive data. In the next stage, the recordings will be transcribed and annotated, allowing the prepared database to be used for training speech-to-text modules and the semantic analysis of statements. The resulting collection will make it possible to create a tool for converting the doctor's speech into text.

The Krakow-based technology company, Proassist, in partnership with scientists from the AGH University, has three years to complete the task. The total value of the project is PLN 11 million.

Other news from this category

Recent news