CORTEX2 innovators: HYMNE's 1st progress update
As digital culture transforms how we experience art and entertainment, a group of Dutch creatives is reimagining the future of live music. Meet HYMNE, a CORTEX2-supported project blending volumetric capture, augmented reality (AR), and extended reality (XR) technologies to create a new kind of concert experience.
Discover how HYMNE is connecting fans and artists like never before — and what’s coming next.
Q: What is HYMNE in one sentence?
A: Hybrid Music and New Experiences (HYMNE) is an innovative project that blends live music with cutting-edge volumetric capture technology to create immersive, hybrid concert experiences that connect audiences and artists in entirely new ways.
Q: What problem are you solving? What makes your solution unique?
A: HYMNE addresses the challenge of making live music more accessible, engaging, and adaptable in a rapidly evolving digital world by combining the energy of in-person performances with immersive, hybrid formats that connect audiences both physically and virtually.
Q: What are HYMNE’s key objectives?
A: The main goals of HYMNE, shared in the first press release about the project, are:
- Reimagine the concert experience: Blend live and digital elements to create hybrid, immersive performances that push the boundaries of traditional music experiences.
- Empower artists: Provide musicians with innovative tools like volumetric capture and mentorship to expand their creative potential and connect with audiences in new ways.
- Increase accessibility: Make high-quality music performances available to broader and more diverse audiences, both in-person and virtually.
- Explore technological frontiers: Pioneer the use of XR, augmented reality, and volumetric technologies to innovate in the entertainment industry.
- Enhance fan engagement: Build deeper connections between artists and audiences through interactive and personalised concert formats.
CORTEX2 support programme progress
Q: What have you achieved so far?
A: The HYMNE project, a collaboration between 4DR Studios, Poppodium Effenaar, and CORTEX2, is now in full swing! Over the next few months, participating artists Anneke van Giersbergen, WITT, CHARLOT, and Dylan van Dael will develop groundbreaking immersive shows that blend live performances with cutting-edge volumetric capture and hybrid concert technology.
Under the mentorship of legendary guitarist Steve Vai, the artists are exploring new ways to connect with audiences through live, augmented, and virtual experiences.
Key milestones:
- March 2025: First try-outs at Effenaar, offering an exclusive preview of the immersive concert concepts.
- May 2025: Grand premiere, where the final hybrid shows will be unveiled.
HYMNE is redefining what live music can be, combining the energy of traditional concerts with innovative digital elements to create a deeper, more interactive connection between artists and fans.
Q: How is participating in CORTEX2 supporting HYMNE?
A: Thanks to CORTEX2, we are able to push creative boundaries and experiment with volumetric capture, augmented reality, and interactive elements, creating truly immersive experiences for fans.
Q: What are your next steps within the programme?
A: We're fully in the process of creating the content for the virtual music experiences, including the volumetric capture recordings with the artists.
Learn more about HYMNE and stay updated on its progress!
Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website:
Open Call 1 winners - Open Call 2 winners
CORTEX2 innovators: FocusVR-ADHD's 1st progress update
Meet FocusVR-ADHD, one of the promising projects selected in the 1st CORTEX2 Open Call. This extended reality (XR) solution is developing an immersive therapy platform that brings patients and therapists together in real-time, regardless of their location. With its unique combination of cognitive training and integrated teleconferencing, FocusVR is rethinking how ADHD therapy is delivered, especially in underserved or remote areas.
Keep reading to discover how FocusVR-ADHD is reshaping digital therapy.
Q: How would you describe FocusVR-ADHD in one sentence?
A: FocusVR is an innovative virtual reality platform designed to improve ADHD therapy by combining immersive cognitive training scenarios with seamless teleconferencing capabilities, fostering deeper engagement and enabling real-time interaction between patients and therapists, regardless of location.
Q: What problem are you solving? What makes your solution unique?
A: FocusVR addresses the challenge of physical distance between therapists and patients, which often limits access to consistent and effective ADHD therapy, particularly in underserved or remote areas.
What sets FocusVR apart is its ability to bridge this gap through a virtual reality platform that combines immersive cognitive training with integrated teleconferencing. This innovative approach enables real-time interaction between patients and therapists, making therapy more accessible, personalised, and engaging, regardless of geographical constraints.
Q: What are FocusVR-ADHD’s main objectives?
A: The key objectives of the FocusVR project include the development of four immersive training scenarios tailored to address core ADHD symptoms such as attention, memory, emotional regulation, and executive functioning. Additionally, the project aims to integrate the advanced CORTEX2 system to enable teleconferencing features, ensuring seamless and real-time interaction between patients and therapists. A crucial part of the initiative is conducting clinical trials to validate the effectiveness of the solution, ensuring it meets the highest standards of therapeutic impact and user experience.
CORTEX2 support programme progress
Q: What have you achieved so far?
A: So far, the FocusVR project has achieved significant progress in several key areas. The team has developed comprehensive UX and UI designs tailored for both therapists and ADHD patients, ensuring the platform is intuitive and user-friendly. Four immersive therapeutic scenarios have been designed, each addressing specific ADHD challenges such as attention, memory, emotional regulation, and executive functioning. These scenarios have been crafted with input from psychological experts to ensure their effectiveness in therapeutic contexts.
The integration of the Rainbow SDK into the Unity engine has also made promising advancements, with initial tests of teleconferencing functionality between therapists and patients proving successful. While this milestone is not yet fully completed, these early trials provide a strong foundation for the next steps in development. Additionally, the implementation of the first therapeutic scenario is well underway, demonstrating the team’s capability to translate conceptual designs into functional, immersive VR experiences.
Q: How is participating in CORTEX2 supporting FocusVR-ADHD?
A: Participating in CORTEX2 has been pivotal for the FocusVR project, particularly through access to the Rainbow SDK. This technology is essential for enabling teleconferencing within the virtual reality environment, allowing real-time interaction between therapists and patients, which is a core feature of our solution. The ability to integrate such advanced communication tools directly into the VR platform has significantly enhanced the project’s potential to make ADHD therapy more accessible and engaging. The support from CORTEX2 has been invaluable in helping us incorporate these critical functionalities into our system.
Q: What are your next steps within the programme?
A: Our next steps within the CORTEX2 programme include the full integration of the Rainbow SDK to enable seamless teleconferencing functionality and secure data handling within the FocusVR platform. Additionally, we aim to complete the full implementation of the first therapeutic scenario and prepare a comprehensive demo for presentation. Following this, we will focus on implementing the remaining three scenarios, each addressing specific ADHD challenges, such as improving attention, emotional regulation, and executive functioning. These will be followed by rigorous clinical testing to evaluate the platform’s therapeutic effectiveness, usability, and impact on patient outcomes. This phase will provide critical insights to refine the system further and ensure it meets the highest standards for deployment in real-world therapeutic settings.
Learn more about FocusVR-ADHD and stay updated on its progress!
Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website:
Open Call 1 winners - Open Call 2 winners
CORTEX2 innovators: CORE-MHC's 1st progress update
Mental health therapy is often limited by distance, stigma, and outdated tools. The CORE-MHC project is exploring how immersive technologies can help remove these barriers by enabling therapists and patients to meet and interact in real time — no matter where they are.
Keep reading to discover how CORE-MHC is using mixed reality (MR) and the CORTEX2 framework to transform remote mental health care.
Q: What is CORE-MHC in one sentence?
A: CORE-MHC aims to facilitate interactive, gamified remote therapeutic experiences between mental health patients and professionals using MR, building upon the CORTEX2 framework.
Q: What problem are you solving? What makes your solution unique?
A: The uniqueness of our solution lies in the fact that it is the first remote therapeutic platform based on mixed reality technologies. This platform enables therapists and patients to participate in remote or in-person therapeutic sessions, utilising real-time data from physiological sensors to gain valuable insights into their psychophysical state. By incorporating gamified MR interactions, it enhances patient engagement and maximises the effectiveness of treatment. The system is designed to improve accessibility to quality mental health care, regardless of the patient's location or available facilities.
Q: What are CORE-MHC’s main objectives?
A: CORE-MHC aims to:
- Help to bridge the treatment gap by making mental health services more accessible and convenient, regardless of the location.
- Boost patient engagement and motivation, and reduce stigmatisation, by incorporating gamification mechanics into the experience.
- Address the user experience challenges in prototypical MR tools by making mental health therapy a more approachable and effective solution for everyone.
- Prevent individual and isolated environments for therapy in XR, where therapists have no ability to guide or interact with their patients.
CORTEX2 support programme progress
Q: What have you achieved so far?
A: The Institute of Computer Technology (ITI) and Fundación SASM have carried out online and in-person co-design sessions to define the use case and collect input from different stakeholders. The team has tested several pilots and demos using the Rainbow platform and the CORTEX2 framework to understand what works best.
Based on this work, they have defined the system architecture and developed an early prototype for an animal therapy session. This allows the therapist to see the patient’s actions in real time while the patient receives live video guidance. Another prototype enables two users, each with a mixed reality headset, to interact in the same virtual session from different locations.
Q: How is participating in CORTEX2 supporting CORE-MHC?
A: Participating in CORTEX2 has allowed us to strengthen the collaboration between ITI and Fundación SASM, working together to define a use-case scenario that could facilitate real therapeutic interventions and situations that mental health professionals encounter in their everyday practice. ITI has gained a better understanding of the mental health domain, while Fundación SASM is actively involved in the definition of requirements of the system and the identification of potential users of their mental health services that could benefit from the integration of this XR solution.
In addition, we have been able to participate in Stereopsia 2024 thanks to CORTEX2, which has helped us to connect with the European landscape of XR professionals, academics and European consortia. This has helped us to disseminate our advances, as well as potential collaborations for future projects.
Q: What are your next steps within the programme?
A: Our next steps will be to finalise the implementation of the gamified therapeutic activity for both PC to headset communication, as well as headset-to-headset communication. Then, we will validate the solution with therapists and patients at Fundación SASM to assess the usability and effectiveness of the remote therapeutic experience in mixed reality.
Learn more about CORE-MHC and stay updated on its progress!
Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website:
Open Call 1 winners - Open Call 2 winners
CORTEX2 innovators: CARE XR's 1st progress update
From wildfires and floods to large-scale accidents, emergency responders need reliable, real-time information to manage crises safely and efficiently. The CARE XR project is exploring how immersive technologies can enhance communication, improve situational awareness, and facilitate faster and more informed decision-making.
Keep reading to learn how CARE XR is working to reshape emergency response — and where it’s headed next.
Q: How would you describe CARE XR in one sentence?
A: CARE XR improves situational awareness during emergency situations.
Q: What problem are you solving? What makes your solution unique?
A: The CARE XR project aims to transform emergency management by addressing the challenges faced by first responders and planners during crises.
Existing systems, like the Next-Generation Incident Command System (NICS), rely heavily on 2D maps and static visualisations, which often fall short in providing the depth of information needed to navigate complex emergencies. Additionally, communication systems are typically one-directional, and the overload of unprocessed data can hinder swift decision-making. These limitations make it harder for emergency teams to respond effectively, risking delays and mismanagement.
The solution offered by CARE XR is unique because it integrates cutting-edge technologies, including extended reality (XR), the internet of things (IoT), and artificial intelligence (AI), into the emergency response process. By replacing 2D maps with immersive 3D visualisations, it gives first responders and planners a much clearer view of the situation. IoT devices collect real-time data, such as temperature or smoke levels, and overlay this information onto the 3D models, making it easier to assess risks and resources. The system also uses machine learning to process large amounts of data quickly, ensuring that decision-makers focus only on the most important information. Moreover, it incorporates voice recognition and natural language processing to enhance communication, allowing responders to exchange critical updates efficiently.
This project stands out not just for its technology but for its vision of collaboration and accessibility. By enabling two-way interaction through XR tools, it empowers responders on the ground to communicate dynamically with command centres. The seamless integration of all these technologies into a unified system ensures faster response times, better resource management, and, ultimately, safer outcomes for both responders and the public. The CARE XR platform has the potential to set a new global standard for emergency management, demonstrating how advanced technologies can save lives and improve safety during critical incidents.
Q: What are CARE XR’s key objectives?
A:
- Enhance emergency response: integrate XR, IoT, and ML with NICS to improve decision-making and resource management during emergencies.
- Develop an XR platform: create a real-time, interactive 3D visualisation tool enriched with IoT data and ML analytics.
- Improve communication: implement advanced systems for real-time audio and video communication using ML and NLP to prioritise critical information.
CORTEX2 support programme progress
Q: What have you achieved so far?
A: We have successfully achieved full integration with Rainbow, enabling the application to send data in various formats, including text, audio, and video. Additionally, it supports the automatic creation of bubbles and the ability to add users to them.
Moreover, we have integrated IoT devices with CORTEX2. These devices can transmit measurements to CORTEX2 endpoints at specified intervals. To facilitate this process, we developed a fully functional Python wrapper, which can also be utilised by other CORTEX2 users working with IoT devices.
Lastly, we have used Scene Reconstruction tools through Gaussian Splatting. At this stage, we can generate point clouds in .ply format. Our next objective is to progress further by generating meshes from these .ply files.
Q: How is participating in CORTEX2 supporting CARE XR?
A: Participation in CORTEX2 has been invaluable and pivotal to the success of our project. We have greatly benefited from the extensive libraries provided by CORTEX2, as well as from the expertise and insights shared by teams that have implemented these libraries, such as DFKI and Intracom. Furthermore, the mentorship and guidance of our mentor have been instrumental in helping us tackle challenges effectively. His support has not only helped us identify solutions but also provided clarity on how to approach complex problems, significantly enhancing our team's progress and confidence.
Q: What are your next steps within the programme?
A: Our next steps within the CORTEX2 programme involve finalising the prototype application and transitioning toward a more stable and robust version. We are preparing to implement a newly designed UI and will also revise the interface for XR headsets to ensure optimal functionality and user experience.
On the technical side, we aim to complete the implementation of speech recognition and summarisation tools, which are yet to be deployed by CORTEX2. To mitigate potential delays, we are also formulating a contingency plan to utilise external libraries for these features if necessary. This dual approach ensures we maintain momentum and continue progressing effectively.
Learn more about CARE XR and stay updated on its progress!
Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website:
Open Call 1 winners - Open Call 2 winners
CORTEX2 innovators: AgriVision's 1st progress update
As the agricultural sector increasingly adopts digital transformation, new tools are emerging to help farmers work more efficiently, sustainably, and collaboratively. AgriVision, a project participating in the CORTEX2 Support Programme, combines advanced extended reality (XR) technology with farm management systems. This initiative is breaking down technological barriers and providing farmers with an accessible, user-friendly platform designed to improve decision-making and boost productivity.
We spoke with the team to know how they are progressing and bringing their vision to life.
Q: How would you define AgriVision?
A: AgriVision is an innovative solution designed to enhance farming efficiency, promote sustainable practices, and make advanced technology accessible to all. By blending cutting-edge augmented and extended reality (AR–XR) technology with a Farm Management Information System (FMIS), AgriVision equips farmers with advanced, yet user-friendly tools that transform decision-making and boost productivity.
Q: What problem are you solving? What makes your solution unique?
A: With AgriVision, farmers can enhance efficiency, optimise resource use, and tackle environmental challenges, while improving profitability.
Q: What are AgriVision’s key objectives?
A:
Overarching objectives:
- Democratise access to remote collaboration tools.
- Apply technology in practical domains.
Specific goals:
- Seamless integration of XR with FMIS and CORTEX2 components.
- Ensure system accessibility and user-friendliness.
- Demonstrate increased efficiency and productivity in farm operations.
- Conduct thorough validation and impact assessment.
- Develop a scalable and replicable model for broader adoption and commercialisation.
CORTEX2 support programme progress
Q: What have you achieved so far?
A:
- Market analysis – Desk research to analyse the market under review.
- Participants' identification – Identifying and reaching out to farmers.
- User requirements – Collecting detailed user stories and feedback.
- Technical specifications – Integration of FMIS and HoloLens.
Q: How is participating in CORTEX2 supporting AgriVision?
A: There are several aspects in which CORTEX2 has supported AgriVision:
- Technical – Through the components provided, continuous communication with the CORTEX2 team, and the expert knowledge shared throughout the project.
- Visibility – Being part of CORTEX2 gives the project credibility and helps promote it to key stakeholders, including policymakers, researchers, and the farming community.
- Collaboration – By engaging with like-minded teams, AgriVision has been able to learn and strengthen its efforts to address real challenges in the farming sector.
Q: What are your next steps within the programme?
A:
- Development of XR and front-end – Connecting with devices through MRTK–Unity, creating 3D objects, front-end access to AgriVision services, and XR development.
- Integration of FMIS, XR, and CORTEX2 – Ensuring seamless integration of all components, followed by testing and resolving any issues.
- Verification of components and product – Verifying individual components and validating the overall solution.
- MVP – Conducting user testing and feedback activities, followed by the release of the AgriVision MVP.
Learn more about AgriVision and stay updated on its progress!
Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website:
Open Call 1 winners - Open Call 2 winners
CORTEX2 innovators: VISOR's 1st progress update
As part of the CORTEX2 Innovators Support Programme, the VISOR team is developing a web-based service designed to make 3D reconstruction from images or videos of small objects easier and more accessible. With a strong focus on automation, quality, and ease of use, VISOR enables users to generate textured 3D meshes that are ready for sharing.
Below, the team shares the inspiring project they’re building and the remarkable journey they are on.
Q: What is VISOR in one sentence?
A: An easy-to-use 3D reconstruction web service that generates optimised textured 3D meshes from images or videos of small objects.
Q: What problem are you solving? What makes your solution unique?
A: VISOR, led by Phasmatic, addresses the challenge of creating high-quality, shareable 3D models of small physical objects from images or videos.
By combining AI-powered 3D reconstruction with automated mesh optimisation, the service delivers high-quality, textured 3D models that are both visually detailed and file-size efficient.
Q: What are VISOR’s main objectives?
A: High-quality 3D reconstruction, ease of use, and shareable 3D textured mesh.
CORTEX2 support programme progress
Q: What have you achieved so far?
A: We have achieved our first milestone of developing an efficient 3D reconstruction pipeline and integrated our solution into a user-friendly web portal, which was our second milestone. This allowed us to validate our service and improve its performance in delivering high-quality, shareable 3D textured meshes.
Q: How is participating in CORTEX2 supporting VISOR?
A: CORTEX2 provided us with the means to create a user-friendly 3D reconstruction web service for single objects. It offered clear KPIs and milestones, which helped shape a well-structured roadmap for the service. Regular meetings with our assigned mentor led to further improvements to the pipeline and helped us stay aligned with the goals of the CORTEX2 programme. It also gave us the opportunity to present and disseminate our service during the Stereopsia event in Brussels.
Q: What are your next steps within the programme?
A: The next step is to validate our web service through collaborations with other teams from the CORTEX2 programme and continue improving the service’s performance.
Learn more about VISOR and stay updated on its progress!
Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website:
Open Call 1 winners - Open Call 2 winners
CORTEX2 innovators: SENSO3D's 1st progress update
We spoke with the team behind SENSO3D, one of the innovators selected in our 1st Open Call. Their project is focused on making the creation of 3D content more accessible, scalable, and efficient, particularly for extended reality (XR) applications. By combining artificial intelligence with a structured 3D object library, SENSO3D is paving the way for faster and more user-friendly virtual scene development, especially in contexts like virtual conferencing and interior design.
Let’s dive into their journey and discover the innovative work they’re developing within the CORTEX2 programme.
Q: What is SENSO3D, in one sentence?
A: SENSO3D is developing an AI-powered 3D object library that transforms 2D images into high-quality, categorised 3D models to revolutionise extended reality (XR) environments and virtual scene creation, particularly for conference room settings.
Q: What problem are you solving, and what makes your solution unique?
A: SENSO3D addresses the challenge of efficiently creating high-quality, categorised 3D models for virtual environments, such as conference rooms, without requiring extensive manual 3D design work. Traditional methods for building XR-ready 3D content are time-consuming, costly, and often demand advanced technical expertise.
Businesses and developers also face challenges such as:
- Limited access to high-quality 3D content that is ready for XR platforms
- Difficulty transforming 2D images into accurate, usable 3D models
- A lack of standardised, categorised libraries for easy model retrieval and integration
SENSO3D offers a cost-effective, scalable solution that automates the 2D-to-3D transformation process and builds an indexed 3D object library. This simplifies workflows for designers, developers, and businesses, helping democratise the creation of 3D content and transforming how virtual environments are designed and experienced.
Q: What are SENSO3D’s main objectives?
A: The key objectives of the SENSO3D project are:
Develop a 3D object library
- Create a library of 3D models representing key elements for conference rooms, such as chairs, tables, lighting, and technology.
- Aim to include around 1,000 models, categorised into specific groups for easier use.
AI tool for 2D-to-3D conversion
- Develop an AI-based tool to convert 2D images into basic 3D models.
- Use available AI techniques to improve accuracy and reduce manual work.
Categorise and organise models: Organise the 3D models into around 50 categories to help users find and use them easily.
Integrate with AR/VR platforms
- Make the models compatible with tools like Unity and the CORTEX2 platform.
- Optimise the models for better performance, such as reducing size and improving loading times.
Provide simple access and usability: Make the library and tools easy to access, including options like a user interface and basic documentation.
Demonstrate use cases: Show how the models and tools can be used in practical applications, such as virtual conference rooms, interior design or planning, and basic AR/VR environments.
Contribute to XR content development
- Support easier and more efficient creation of 3D content for AR/VR applications.
- Focus on providing resources that can be useful for developers and businesses.
These objectives focus on building a simple and functional 3D model library and tools while supporting basic AR/VR applications without overpromising on outcomes.
CORTEX2 support programme progress
https://youtu.be/I7buG3XWWog
Q: What have you achieved so far?
A:
3D object library development
- Collected and categorised 3D models for conference room items, including chairs, tables, and technology equipment.
- Scanned and processed over 100 real-world objects into usable 3D models.
AI tool development
- Developed and tested the initial version of the 2D-to-3D conversion tool, which can generate 3D models from images.
- Demonstrated the tool with basic functionality for detecting and creating simple 3D objects like chairs.
Dataset integration: Integrated relevant public datasets, such as ShapeNet and Matterport3D, to support AI training and improve model accuracy.
Categorisation of 3D Models: Organised models into 50 categories for easier management and retrieval.
Initial demonstrator: Created a basic demo platform to showcase the 2D-to-3D conversion tool and its output for selected objects.
Performance optimisation: Began implementing methods like polygon count reduction and texture optimisation to prepare the models for AR/VR environments.
The impact we have achieved so far
- Provided a foundation for 3D model standardisation and integration into AR/VR platforms.
- Created a starting point for automating 2D image to 3D model workflows.
- Demonstrated initial use cases, such as simple virtual conference room elements.
These milestones represent steady progress toward building a functional 3D object library and AI tools, aligning with the project goals.
Q: How is participating in CORTEX2 supporting SENSO3D?
A:
Participating in the SENSO3D has provided significant value to our project in the following ways:
Access to mentorship and guidance: The mentorship from CORTEX2 has helped us define the essential "must-have" elements for SENSO3D, such as key items like chairs, tables, and technology equipment for the 3D object library. Regular feedback and support from mentors have allowed us to refine our approach and align our outputs with project goals.
Resource identification and partnerships: With guidance from the programme, we identified suitable sources and partners to acquire modifiable 3D parts and integrate them into our library. This includes online resources and scanning partnerships
Validation of the AI tool: The programme provided a framework to test and demonstrate our AI-powered tool. This has helped us showcase a working prototype and validate its functionality for selected categories.
Structured project monitoring: The CORTEX2 programme's structured timeline and monitoring processes have ensured that we stay on track with deliverables and milestones. Regular surveys and reporting have provided clarity on progress and areas that need improvement.
Networking and collaboration: The opportunity to connect with other funded projects and partners has opened avenues for potential collaborations, such as access to additional datasets or 3D scanning resources.
Support in overcoming challenges: CORTEX2 has been instrumental in helping us address project obstacles, such as resource constraints for AI training and challenges with sourcing real-world items for 3D scanning. For instance, their support in connecting with academic institutions or retailers has been valuable.
Most valuable aspects for our team: The programme’s mentorship and technical guidance, support for accessing resources, datasets, and feedback loops for validation, and structured monitoring that helps us measure progress and address challenges early on.
Participating in CORTEX2 has helped us establish a strong foundation for developing our 3D object library and AI tools, while providing the necessary resources and guidance to improve our project outcomes.
Q: What are your next steps within the programme?
A:
Expand the 3D object library
- Continue scanning and processing real-world objects, focusing on conference room components like tables, chairs, lighting, and technology.
- Incorporate additional commercially available 3D models to meet the target of 1,000 categorised models.
Enhance the AI tool
- Improve the 2D-to-3D conversion tool by expanding its detection capabilities from 4-5 categories to at least 10 key categories through additional datasets and AI model training.
- Implement optimisation techniques to increase accuracy and efficiency, such as transfer learning, to reduce resource requirements.
Develop use cases and demonstrators
- Finalise the demo platform showcasing AI-generated 3D models from 2D images.
- Create full virtual conference room scenes with categorised and optimised 3D objects for AR/VR use cases.
Optimise and integrate models
- Optimise 3D models for performance using techniques like polygon reduction, texture compression, and occlusion culling.
- Integrate the library and AI tools with Unity and the CORTEX2 platform for testing and validation.
Prepare for final validation
- Ensure deliverables and KPIs are met for Sprint 2 and Sprint 3, including model integration, AI tool deployment, and performance testing.
- Collect user feedback to identify areas for final improvements.
Learn more about SENSO3D and stay updated on its progress!
Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website:
Open Call 1 winners - Open Call 2 winners
CORTEX2 innovators: TIP's 1st progress update
As extended reality (XR) technologies become increasingly accessible, the demand for high-quality, adaptable digital assets is growing fast. The Infinity Palette (TIP) rises to the challenge with a mission to deliver a versatile and scalable library of 2D and 3D assets for XR developers and creators. TIP is one of the projects selected through the CORTEX2 programme to explore new solutions that simplify the creation and adoption of XR. Their goal? To make it easier for users to build immersive virtual environments tailored to the needs of education, culture, and entertainment.
Let’s dive into their journey and discover the innovative work they’re developing within the CORTEX2 programme.
Q: How would you describe TIP in one sentence?
A: The Infinity Palette (TIP)’s mission is to provide a comprehensive and scalable library of 2D and 3D assets that CORTEX2 users can easily implement.
Q: What problem are you solving, and what makes your solution unique?
A: With over a decade of experience in creating immersive XR environments, the 3D Interactive team will leverage its vast asset pool to meet the diverse needs of the CORTEX2 platform.
At the core of TIP’s vision is the creation of highly optimised assets, ensuring broad accessibility and ease of integration. These assets, encompassing both abstract and environment-specific 3D models, will provide users with a solid foundation for building rich virtual experiences.
TIP’s approach emphasises flexibility and customisation, offering users the ability to modify and adapt assets to their specific needs.
Through these efforts, The Infinity Palette project seeks to expand the capabilities of the CORTEX2 platform by providing users with versatile, immersive virtual environments that can be adapted for various educational, cultural, and entertainment purposes.
Q: What are TIP’s main objectives?
A: The primary objective of TIP is to design and develop a comprehensive 3D asset library that supports the creation of immersive environments for the education, entertainment & and culture domains within the CORTEX2 platform.
CORTEX2 support programme progress
Q: What have you achieved so far?
A: During the project’s latest phase, we began by creating the first set of assets and assembling initial Unity packages, in line with the project plan and tasks defined previously. Our key developments and achievements in this phase include:
- Design block out-of-the-scenes
- Asset creation of static environments (replacing block out)
- Asset creation of dynamic objects
- Different unique functionalities for different scenes
- Exported a whole scene as a Unity package to other CORTEX2 users
Q: What are your next steps within the programme?
A: Next, we will complete the remaining CORTEX2 user testing based on existing connections and begin collaborating with other CORTEX2 users, including HYMNE and SENSO3D. This phase will also involve refining our work based on user feedback and delivering the final version of the project.
Learn more about TIP and stay updated on its progress!
Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website:
Open Call 1 winners - Open Call 2 winners
CORTEX2 innovators: MHI's 1st progress update
The Multiplayer Haptic Interactions (MHI) project, supported by CORTEX2, is pushing the boundaries of realism and collaboration in extended reality (XR). Developed by SenseGlove, it introduces seamless integration of haptic feedback and hand tracking into shared immersive environments, helping users interact, collaborate, and connect in XR like never before.
Let's hear from the team about their latest progress.
Q: What is MHI, in one sentence?
A: With the MHI project, SenseGlove develops an API that seamlessly integrates haptic gloves and hand tracking into a single scene, enabling collaborative and realistic interactions in XR.
Q: What problem are you solving, and what makes your solution unique?
A: Our solution addresses the fragmentation in XR interactions by integrating haptic gloves and hand tracking into a single API. This unique approach enables realistic tactile feedback and precise hand movements, fostering natural, immersive, and collaborative experiences in shared XR environments — ideal for applications like training, design, and remote teamwork.
Q: What are MHI’s main objectives?
A: Our primary objective is to develop a Unity SDK that supports integrating at least four users into a single scene, accompanied by a reference implementation featuring 10 sample interactions, including person-to-person interaction.
CORTEX2 support programme progress
Q: What have you achieved so far?
A: We have integrated OpenXR hand tracking into the SenseGlove Interaction and Haptics SDK, creating a template scene where users wearing haptic gloves and using hand tracking can interact with objects, high-five, and shake hands. This scene is designed for standalone HMDs, enabling immersive and collaborative experiences.
https://www.youtube.com/watch?v=ZYN9bx7fTzg
Q: What are your next steps within the programme?
A: During the final sprint, we aim to optimise the scenario, build a scene with four users, and make the setup easier and more user-friendly.
Learn more about MHI and stay updated on its progress!
Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website:
Open Call 1 winners - Open Call 2 winners
CORTEX2 innovators: MGL's 1st progress update
In the rapidly evolving world of extended reality (XR), intuitive interaction is key. One of the innovators in CORTEX2’s support programme is MGL, a project that is pushing gesture recognition to the next level. It offers developers a high-precision toolkit for recognising hand gestures — enabling more natural, seamless interaction in immersive environments.
Learn more about their work and how CORTEX2 is helping them bring their vision to life.
Q: How would you describe MGL in one sentence?
A: The Magos Gestures Library (MGL), led by QUANTA & QUALIA, enables gesture recognition for intuitive interactions in XR environments.
Q: What problem are you solving, and what makes your solution unique?
A: Current hand-tracking solutions often fall short when it comes to precision, flexibility, and integration. They can feel clunky, unreliable, or limited to predefined gestures — posing challenges for developers and users alike.
Our solution changes that. MGL delivers millimetre-accurate tracking of finger joint movements, supports dynamic customisation of gestures, and integrates seamlessly with the CORTEX2 framework. That means more natural, personalised, and precise interactions, tailored to different use cases and user needs.
Q: What are the key objectives of MGL?
A: We are focused on three core goals:
-
Gesture recognition module development: Build a module that recognises at least 15 foundational gestures (like thumbs up, pinch, wave, and fist), offering reliable and intuitive gesture-based interaction in XR.
-
Customisation and extensibility: Enable developers to dynamically add and customise gestures, providing flexibility for diverse applications.
-
Seamless integration: Ensure smooth integration into the CORTEX2 framework, making MGL scalable and compatible with a range of XR scenarios.
CORTEX2 support programme progress
Q: What have you achieved so far?
A: We have already hit some significant milestones:
-
Completed a full library of foundational gestures — 15 in total, covering core actions like pinch, fist, and thumbs up.
-
Enabled dynamic gesture customisation, so that developers can tailor interactions to their specific use cases.
-
Built a Unity demo scene, offering a hands-on demonstration of all available gestures in action.
These steps not only showcase the system’s capabilities but also provide a flexible, user-friendly solution for XR environments, enhancing usability and interaction design.
Q: How is participating in CORTEX2 supporting MGL?
A: The programme has given us far more than just funding.
-
Mentorship and collaboration: CORTEX2 connects us with technical experts and a broader community of innovators. The guidance and feedback we have received have helped us solve challenges and refine our product.
-
Visibility and impact: Being part of the ecosystem opens doors—to new partnerships, wider audiences, and real-world adoption opportunities.
This support is helping us fast-track development and move closer to real-world deployment.
Q: What are your next steps within the CORTEX2 programme?
A: We are heading into the next phase of the programme with clear priorities:
-
Final integration of MGL into the CORTEX2 framework to ensure seamless compatibility.
-
Comprehensive validation, including extensive testing of gesture accuracy and reliability in real-world settings.
-
Demonstration and refinement: We will showcase the library in industrial applications, collect feedback, and continue to improve the system.
-
Collaboration and knowledge sharing: We will keep working closely with CORTEX2 partners to align objectives and spark further innovation.
MGL is setting a new standard for gesture-based interaction in XR — precision, personalisation, and plug-and-play readiness all in one. Stay tuned as they bring the power of natural gestures to immersive experiences across industries.
Learn more about MGL and stay updated on its progress!
Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website:
Open Call 1 winners - Open Call 2 winners
CORTEX2 innovators: FLYTEX's 1st progress update
FLYTEX, one of the innovative projects selected through CORTEX2’s Open Call 1, is bridging the gap between agriculture and immersive technologies by integrating IoT data into extended reality (XR) environments. The team is on a mission to empower the agri-food sector — especially small farms — through smarter, more efficient, and sustainable tech solutions.
Q: How would you describe FLYTEX in one sentence?
A: The FLYTEX project, led by FlyThings Technologies, aims to transform videoconferences and immersive environments by providing access to IoT sensor data, using the agricultural sector to validate our results.
Q: What problem are you solving? What makes your solution unique?
A: Although the agri-food sector’s digitalisation has been increasing in recent years, XR applications remain underutilised — particularly by small farms. Our solution aims to change this by making the industry more technologically advanced, efficient, and sustainable, ultimately improving decision-making and resource management.
Q: What are the key objectives of your project?
A: Our main goal is to create an innovative, data-driven environment that aids in the advancement of the agri-food sector through the application of IoT and XR technologies, integrating our IoT platform.
CORTEX2 support programme progress
Q: What have you achieved so far?
A: We have developed an API CORTEX2 client and integrated it into the Flythings IoT platform. The system can now connect to a wide variety of devices using different protocols, making our solution versatile and scalable.
Q: How is participating in CORTEX2 supporting FLYTEX?
A: The CORTEX2 team has provided expert advice and support to help us overcome minor technical issues during the integration of both platforms. Their guidance has been key to our progress.
Q: What are your next steps within the CORTEX2 programme?
A: We are focused on finalising the user interface and completing integration with one of the CORTEX2 pilots to demonstrate our solution in action.
Learn more about FLYTEX and stay updated on its progress!
Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website:
Open Call 1 winners - Open Call 2 winners
CORTEX2 innovators: EXTERNALISE's 1st progress update
The EXTERNALISE project, selected as one of the winners of the CORTEX2 Open Call 1, is pushing the boundaries of motion capture and embodiment in virtual environments — creating real-time, multi-user experiences with full-body avatars, without the need for suits or specialised hardware.
The project is led by MOVERSE. Find out more about their progress and upcoming plans.
Q: What is EXTERNALISE, in one sentence?
A: A real-time, multi-user, markerless motion capture and full-body avatar embodiment.
Q: What problem are you solving? What makes your solution unique?
A: We are focusing on capturing the expressive motion of multiple users in real time across different locations and transmitting them into a shared virtual space with their full-body avatars. The main advantage of our solution is that it is frictionless, as it does not require any suits or high-end setups.
Q: What are EXTERNALISE's key objectives?
A: To achieve its goal, EXTERNALISE needs to scale markerless MoCap technology to support multiple users in real time and integrate client-side technology that can receive and render the users’ avatars.
CORTEX2 support programme progress
Q: What have you achieved so far?
A: The team has demonstrated live, markerless motion capture at room scale for up to four subjects. The developed system supports multiple nodes connecting to a shared virtual space, allowing users to collaborate and communicate using real-time, expressive avatar representations.
https://www.youtube.com/watch?v=ELkAILZm-EQ
Q: How is participating in CORTEX2 supporting EXTERNALISE?
A: CORTEX2's support has been instrumental in our progress and in achieving our goals. The opportunity itself, combined with insightful guidance from mentors and the timely coordination of activities, has created a highly productive environment for our team.
Q: What are your next steps within the CORTEX2 programme?
A: The core technology has been developed and successfully demonstrated live. Our next steps will focus on system refinements, testing, validation, and dissemination of the technology.
Learn more about EXTERNALISE and stay updated on its progress!
Want to explore more extended reality (XR) innovation? Browse all our supported projects on the CORTEX2 website: