Gesture control is one of the most intuitive ways to interact in extended reality (XR) — but making it work reliably across different users, devices, and platforms is a challenge. That is where CDLPG comes in. Supported by CORTEX2, this project is developing a scalable system for personalised gesture recognition that integrates directly with the CORTEX2 framework.
We spoke with the team to learn more about their vision, progress, and next steps.
Q: What is CDLPG, in one sentence?
A: The CDLPG project (Co-development of a Dynamic Library of Personalised Gestures), led by Sensorama Lab, aims to create a dynamic, scalable gesture recognition system integrated with CORTEX2, enabling personalised hand and body gestures to trigger predefined actions.
Q: What problem are you solving, and what makes your solution unique?
A: We are tackling a key challenge in the AR/VR space: creating a gesture recognition module that can adapt to various use cases within the CORTEX2 framework. Most existing systems struggle with recognising a wide range of specific gestures — especially when performed differently by different users. That impacts consistency and usability.
We are also working to overcome the technical challenges of integrating gesture recognition with diverse AR/VR hardware. Our solution is designed to work across platforms and devices, making it more accessible and reliable.
Q: What are CDLPG’s main objectives?
- Develop gesture recognition: Accurately capture and interpret hand and body gestures.
- Integrate with CORTEX2: Ensure compatibility with AR/VR hardware and the CORTEX2 framework.
- Enable gesture-to-action mapping: Link gestures to predefined actions within applications.
CORTEX2 support programme progress
Q: What have you achieved so far?
A: We have developed a system that connects input data with gesture recognition components and created a functional prototype where gestures are processed and recognised. Here is a short demo of the CDLPG prototype:
An operational gesture registration system has been built, capable of recognising at least seven distinct gestures. The system has been integrated with one of the CORTEX2 pilots, allowing compatibility with its framework. Additionally, we have enabled gesture-to-action mapping and integrated a dynamic action mapping component within the CORTEX2 pilot.
We expect that our solution will bring about the following impact:
- Enhanced user interaction: By implementing this solution, we aim to transform user engagement within AR/VR realms. Our approach centers on integrating seamlessly intuitive, gesture-driven interfaces, redefining the way users navigate and interact within these digital spaces. This shift towards more organic and user-friendly controls is set to elevate the overall immersion and accessibility of AR/VR experiences.
- Increased versatility of AR/VR applications: With a dynamic library of gestures, AR/VR applications can be tailored to a wide range of uses, from collaboration to professional training and education.
Q: How is participating in CORTEX2 supporting CDLPG?
A: Participating in the CORTEX2 program has been very valuable for our team. As a Ukrainian company, being part of a joint European XR platform allows us to collaborate with international partners and contribute to a shared project. This opportunity enables us to exchange knowledge and experience with our European counterparts, helping us improve and grow. Working within CORTEX2 also supports the development and integration of our gesture recognition system, with valuable input and feedback from the wider network. We are proud to be part of this collaborative effort.
Q: What is next for CDLPG?
A: Next, we will focus on testing, refining, and promoting the system. We will develop a checklist to test the system’s reliability and conduct thorough evaluations. Focus groups and usability studies will be organised to gather user feedback, which will help us improve performance and usability. Detailed documentation of the software module will be prepared, including mentor feedback, and published on GitHub. Finally, we will create a promotional video to showcase the project and share its progress.
Learn more about CDLPG and stay updated on its progress!
Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website: