2025, January 30
Share this articleNU ISSAI intends to create a multimodal sparse model inspired from the DeepSeek architecture capable of processing images, text, and audio.
The recent release of two AI models developed by the Chinese company DeepSeek has been hailed as a major breakthrough in artificial intelligence. These technologies, which surpass OpenAI's models in some aspects, are openly available and challenge big tech companies worldwide. The Institute of Smart Systems and Artificial Intelligence ISSAI at Nazarbayev University (NU) is also exploring these models to develop AI solutions customized to the needs of Kazakhstan. Dr. Atakan Varol, General Director of ISSAI, emphasized that adapting the latest high-performance AI models for specific tasks—such as improving Kazakh language comprehension or fine-tuning them for specialized domains—requires independent additional training. One of the challenges he highlighted was that DeepSeek had not disclosed its fine-tuning methodology.
"Right now, many research groups around the world are trying to figure out how to train DeepSeek-R1 model further, how to create better models or customized models based on this, and the ISSAI NU research team is also striving for this objective; our researchers are deciphering the downloaded model block by block, understanding how it works, and planning to train the next generation of ISSAI generative AI models, using this architecture," noted Dr. Varol.
Recently, ISSAI introduced Oylan, a pilot-stage language-vision model that, unlike KAZ-LLM, which processes only text queries, can work with both images and text. The experience gained in developing KAZ-LLM and Oylan models will be crucial in creating a universal multimodal AI system.
"We expect that our multimodal sparse generative AI model will utilize the efficient architecture of DeepSeek, the so-called Mixture of Experts Architecture, but right now, DeepSeek-R1 can only process text to text. If we can find the resources, we envision creating a model that will be able to understand images, audio, and text, and also generate these three modalities. This way, we will create a universal multimodal generative AI tool that can process any input modality and generate any output modality. This is, right now, the Holy Grail of race in AI," explained Professor Varol.
However, developing advanced AI models is not enough—additional computing power is essential for making them accessible to a wider audience for usage. Currently, the pilot version of Oylan runs on a single NVIDIA A100 server, limiting simultaneous usage to just 20 users.
"Our objective is first to obtain resources for our Institute to cover computational resources and operational expenses, and afterward we plan actively compete in the global AI race and create our multimodal sparse model, which brings all different modalities such as image, text, and audio, and uses the efficient mixture-of-experts architecture. This will allow us to move closer to creating Artificial General Intelligence (AGI) while lowering deployment costs. However, even if we succeed in building a model that addresses part of this grand challenge, a critical issue remains: the need for extensive computing power to further train and deploy such models," added Professor Varol.
Creation of KAZLLM by ISSAI researchers allowed Kazakhstan to enter a select group of around 20 countries which have created large language models tailored for their linguistic and cultural needs. With the recent introduction of Oylan, Kazakhstan became one of the dozen countries who have the technical know-how and intellectual workforce to develop advanced language-vision AI models. Later this year, sparse multimodal AI models capable of seamlessly processing images, text, and audio are expected to be introduced. If Kazakhstan becomes one of the nations pioneering this technology, it will have a strong position in securing its digital sovereignty and increasing its competitiveness in the digital field.
