In an era defined by rapid technological advancements, Institute of Smart Systems and Artificial Intelligence (ISSAI) of NU stands at the brink of a new frontier in human-computer interaction in Kazakh language. The next breakthrough in this direction is the launch of the first Digital Human Prototype “UMAY”.
Umay represents a virtual assistant proficient in the Kazakh language, embodied in a digitally crafted, lifelike human avatar. Its capabilities span a diverse range of domains, including science, education, history, technology, and medicine.
The avatar is symbolically named after the mother goddess from Turkic mythology. Firstly, the name is related to Kazakh culture, and secondly, Umay is the first digital human prototype and can be considered the mother of future digital humans ISSAI will develop.
“Our main objective was to show the advanced capabilities of the Kazakhstani intellectual workforce (ISSAI team) and demonstrate “We can do it in Kazakhstan.” This way, we intended to convince different stakeholders to allocate resources into science and research. UMAY uses a number of key technologies developed by our Institute since September 2019. These include text-to-speech, automatic speech recognition, neural machine translation, facial AI, prompt engineering, and photorealistic avatars.” – said Doctor Atakan Varol, ISSAI director, department chair and professor at NU’s School of Engineering and Digital Sciences.
According to Dr. Varol, the main technical challenge for the project was systems integration, meaning the team had to get multiple AI systems running at the same time. At the heart of this system lies ChatGPT from OpenAI, although it is acknowledged for its less-than-optimal performance with the Kazakh language. To overcome this limitation, Umay harnesses an advanced neural machine translation model, Tilmash, also developed by ISSAI, which seamlessly translates user queries into English and subsequently transforms the responses of ChatGPT back into the Kazakh language.
“Our institute previously created a two-way machine translation system into 4 languages called Tilmash. This model, compiled from a large text base in different styles, can compete with Google and Yandex translators. The model translates queries voiced in Kazakh into English, and then translates responses from ChatGPT from English into Kazakh,” explains one of the Umay developers, ISSAI senior data analyst Askat Kuzdeuov.
Powered by Unreal Engine 5, Umay excels in delivering high-standard, real-time facial animations. It integrates cutting-edge speech technologies developed by ISSAI, including automatic speech recognition (ASR) and text-to-speech (TTS), to facilitate natural and fluent communication in the Kazakh language.
With further development, Umay can complement the human workforce for increased efficiency and decreased costs in numerous areas. . For further customization and applications, UMAY needs its own large language model, i.e., the Kazakh artificial mind. Large language model training requires huge data storage and enormous computational resources. This is only the beginning of a long journey that requires support from the state and society, NU analysts note.
Here you can see how the digital prototype works.
Institute of Smart Systems and Artificial Intelligence (ISSAI) was founded in September 2019 to serve as the driver of research and innovation in the digital sphere of Kazakhstan with the focus on AI research. Located at the C4 Research Building of NU, ISSAI conducts interdisciplinary research on machine intelligence for solving real-world problems of industry and society. ISSAI provides an agile framework for research, innovation and collaboration with national and international partners in education, industry and government and contributes to the digital ecosystem of Kazakhstan in the advancement of national development goals.
