Q: What is NITRUS in one sentence?
A: NITRUS is an advanced AI gesture recognition system that helps people interact with machines and virtual environments.
Q: What problem are you solving? What makes your solution unique?
A: Gesture recognition pipelines usually require a lot of computing resources and effort to make them work with a limited set of gestures. Our End-to-End AI pipeline processes video instead of single frames, working in real time and constrained resources with high precision.
Q: What are NITRUS’s main objectives?
A:
- To create a solution that enriches the interaction of people’s digital experiences by interpreting and identifying human gestures.
- To gather and/or create gesture datasets with variability and inclusion considerations that can be used for the development and validation of said solution.
- To develop an action recognition AI model based on Spatio-temporal CNNs capable of identifying people’s actions in real-time.
- To provide tools for the optimisation and deployment of this model in hardware-constrained scenarios while maintaining real-time and bandwidth requirements.
CORTEX2 support programme progress
Q: What were the main activities implemented and milestones achieved during Sprint 1 of the CORTEX2 Support Programme?
A: In sprint 1, we have defined the requirements and specifications of our solution as well as familiarised ourselves with CORTEX, the state of the art and the available datasets. We have implemented and tested ~ 10 different AI models that fulfil one or more of the steps needed for the gesture recognition pipeline, covering two completely different approaches that may be used for the application.
Q: What have you achieved so far?
A: The current prototype of the NITRUS solution fulfils all the original expectations. The technical objectives and KPIs have been achieved, and the project is on track for the final stage of the implementation.
The current prototype is able to detect 14 different gestures with 92% accuracy. It can be executed in CPU and GPU, making it compatible with resource-constrained devices, and it works with a ~270ms latency and ~ 2s response time.
Q: How is participating in CORTEX2 supporting NITRUS?
A: CORTEX2 has supported us in the development of the technology and provided a way to integrate our technology into teleconference applications.
Our mentors have given us interesting feedback from both the technology and use case perspectives, helping us to create a solution that is modular and adaptable to many scenarios.
Q: What are your next steps within the CORTEX2 Programme?
A: Fully validate the technology and exploit the results in other gesture recognition applications.
Learn more about NITRUS and stay updated on its progress!
Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website: