admin

CORTEX2 innovators: MHI's 3rd progress update

MHI is advancing multiplayer haptics for immersive extended reality (XR) training. In Sprint 3 of the CORTEX2 Support Programme, they showcased seamless interaction with hand tracking and haptic gloves across devices and borders.

Read on to learn about MHI's latest breakthroughs.

MHI's progress during Sprint 3 of the CORTEX2 Programme

Q: How would you summarise MHI's latest developments during Sprint 3 of the CORTEX2 programme?

A: During the CORTEX2 Programme, SenseGlove successfully developed and validated a robust Multiplayer Haptic Interaction System tailored for XR training applications. This system enables seamless integration of both OpenXR hand tracking and SenseGlove Nova 2 haptic gloves within a single XR framework. Users can effortlessly switch between hand tracking and haptic gloves depending on the training scenario, offering unmatched flexibility and accessibility in XR interaction design.

Key technical achievements include the development of a fully modular Unity-based interaction toolkit, supporting a wide range of interactive elements such as buttons, toggles, dials, drawers, and complex two-user social interactions like handshakes and high-fives. The toolkit was enhanced with built-in multiplayer functionality using the Mirror networking framework, enabling real-time multi-user interaction across both standalone and PCVR platforms.

We successfully demonstrated stable, cross-device multiplayer sessions with up to four users, maintaining a consistent 60Hz frame rate on standalone headsets like the Meta Quest 3. The system was tested extensively, including remote, cross-border sessions, confirming both technical robustness and geographic scalability.

A reference scene featuring twelve fully interactive training modules was developed as part of a developer-oriented SDK. Furthermore, we designed and validated a turnkey multiplayer training package that integrates all hardware and software components into a single deployable solution — drastically reducing setup time and technical complexity for end users in industrial and educational contexts.

This collaboration between SenseGlove and CORTEX2 highlights Europe’s leadership in haptic technologies and high-fidelity multiplayer XR training, with impactful applications in remote education, technical instruction, and virtual collaboration across industries.

Check out MHI's previous interviews and stay updated on its progress!

Want to know more about other CORTEX2 innovators' updates? Browse all our supported teams on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: EXTERNALISE's 3rd progress update

EXTERNALISE is redefining real-time communication with multi-user, markerless motion capture. In Sprint 3 of the CORTEX2 Support Programme, they demonstrated immersive avatar interaction across distributed nodes.

Read on to learn about EXTERNALISE's latest breakthroughs — and what’s next!

EXTERNALISE's progress during Sprint 3 of the CORTEX2 Programme

Q: How would you summarise EXTERNALISE's latest developments during Sprint 3 of the CORTEX2 programme?

A: For this sprint, we demonstrated multi-user multi-node markerless motion capture using the new version of the Moverse Capture Studio that was previously extended to support multiple actors' captures in real-time.

Q: What milestones did EXTERNALISE reach during Sprint 3, and what impact do they have?

A: Our demonstration involved two distributed nodes, with a system deployed on each one to capture a group of users, specifically two users for each node. All users were made virtually present in a shared environment, with a fifth user joining with a headset and participating while spectating the four physically remote users interact. This demonstration showed the feasibility of multi-user communication and interaction with embodied avatars mimicking their real-time behaviours, gestures and actions. This technology can drive future communications that extend beyond traditional video and audio, driving expressive avatars and conveying non-verbal communication cues efficiently and effectively.

Another key milestone in this third sprint was the live demonstration of multi-user live capture at this year’s Game Developers Conference (GDC25). Showcasing the technology in a gamified context attracted significant interest as a technology to drive future real-time interactive content for entertainment and advertising, among others.

https://www.youtube.com/watch?v=9hvII6z9ShE

Q: What are EXTERNALISE's next steps?

A: We have already released these developments into our core product, the Moverse Capture Studio, and our goal moving forward is to continue improving the technology, its downstream integrations and support all the creatives using it. We are also seeking to partner with enterprises that aim to integrate it into future communication and interactive XR platforms.

Check out EXTERNALISE's previous interviews and stay updated on its progress!

Want to know more about other CORTEX2 innovators' updates? Browse all our supported teams on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: MGL's 3rd progress update

Quanta & Qualia concluded Sprint 3 of the CORTEX2 Support Programme with a powerful upgrade to extended reality (XR) interaction through their Magos Gestures Library (MGL) project. Integrated into an industrial training scenario, MGL enables precise, controller-free hand gesture control with haptic gloves.

Read on to learn about MGL's latest breakthroughs.

MGL's progress during Sprint 3 of the CORTEX2 Programme

Q: How would you summarise ARY's latest developments during Sprint 3 of the CORTEX2 programme?

A: The Magos Gestures Library (MGL) project, developed by Quanta & Qualia, aimed to improve how users interact within XR environments by enabling natural, real-time hand gesture recognition using Magos haptic gloves. Through the CORTEX2 Support Programme, the MGL module was developed as a Unity-integrated tool that allows users to recognise, create, and customise hand gestures with high precision based on application needs.

In the final sprint, MGL was successfully integrated into the SELFEX2 project, one of the winners of the 1st CORTEX2 Open Call, where it enabled intuitive and immersive training for industrial assembly tasks. By replacing traditional VR controllers with precise gesture-based interaction, MGL improved the realism, usability, and effectiveness of the training experience by enabling users to use Magos gloves without having to use the controllers for interactions with UIs or menus at the same time. Still, all these functionalities have been converted to gestures, so the end user can call the menu based on a specific hand gesture, such as thumb up, or teleporting via pinch gesture and so on. A key innovation was the introduction of a calibration mechanism that adapts to different hand sizes, ensuring consistent tracking accuracy for all users.

Validated by technical partners and praised by end users, including continued interest from CTAG (SELFEX2 partner), the MGL project has demonstrated strong potential for impact in XR applications and beyond.

Check out MGL's previous interviews and stay updated on its progress!

Want to know more about other CORTEX2 innovators' updates? Browse all our supported teams on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: TIP's 3rd progress update

With The Infinity Palette (TIP), users can access a growing library of customisable 3D assets and virtual environments for education, culture and entertainment. In Sprint 3, TIP advanced cross-platform integration and design flexibility to enhance immersive experiences on the CORTEX2 platform.

Read on to learn about TIP's latest breakthroughs.

TIP's progress during Sprint 3 of the CORTEX2 Programme

Q: How would you summarise TIP's latest developments in the CORTEX2 programme?

A: The Infinity Palette (TIP) aims to develop a comprehensive 3D objects library and customisable virtual environments designed for educational, cultural and entertainment applications. Its mission is to deliver cross-platform assets optimised for Unity, providing CORTEX2 Users with dynamic and static 3D objects that can easily be integrated into immersive learning and entertainment experiences.

TIP’s objectives focus on creating versatile virtual spaces, including classrooms, lobbies, group study areas and virtual concert venues. These environments are designed to enhance user engagement by offering flexible customisation options and entertainment experiences.

The project leverages a decade of experience in extended reality (XR) content creation to ensure the assets not only meet aesthetic and technical standards, but also promote seamless integration across different platforms, allowing a wide range of users to benefit from the CORTEX2 platform. The project’s technical approach is structured in three phases: conceptual, development and refinement.

These phases ensure that all assets are optimised and ready for deployment, providing educators, cultural institutions and entertainment venues with tools to create engaging, interactive and immersive experiences.

TIP’s long-term vision aligns with the CORTEX2 goal of fostering open access and innovation across European digital ecosystems by expanding the platform’s user base and supporting the broader adoption of Virtual World’s technologies.

Check out TIP's previous interviews and stay updated on its progress!

Want to know more about other CORTEX2 innovators' updates? Browse all our supported teams on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: CDLPG's 3rd progress update

CDLPG wraps up Sprint 3 of the CORTEX2 Support Programme with a validated, user-tested gesture recognition system. Enhanced accuracy, flexibility, and responsiveness now pave the way for broader adoption and integration.

Read on to learn about CDLPG's latest breakthroughs — and what’s next!

CDLPG's progress during Sprint 3 of the CORTEX2 Programme

Q: How would you summarise CDLPG's latest developments during Sprint 3 of the CORTEX2 programme?

A: During Sprint 3, we focused on systematically validating and finalising the gesture recognition system. Key activities included comprehensive testing and validation of integrated components, preparation of detailed technical documentation, and performance optimisation based on user and QA feedback. A structured checklist and insights from usability studies guided the evaluation process, ensuring accuracy, reliability, and an enhanced user experience.

Q: What milestones did CDLPG reach during Sprint 3, and what impact do they have?

A: Key milestones in Sprint 3 included the successful validation of the gesture recognition module, completion of usability studies, and integration of user and QA feedback. These efforts led to significant improvements in recognition accuracy and responsiveness. The module now reliably supports both predefined and user-defined gestures, enhancing flexibility and usability within the CORTEX2 framework.

https://www.youtube.com/watch?v=h2Jo_als2y4

Q: What are CDLPG's next steps?

A: Our next steps involve publishing the gesture recognition module on the Unity Asset Store, making it accessible for broader adoption. We will also actively disseminate our results through presentations and publications, and continue engaging with other CORTEX2 partners to explore integration opportunities and collaborative use cases.

Check out CDLPG's previous interviews and stay updated on its progress!

Want to know more about other CORTEX2 innovators' updates? Browse all our supported teams on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: ARY's 3rd progress update

ARY closes Sprint 3 of the CORTEX2 Support Programme with a key technical milestone: a new Ultra-Wideband tag that makes extended reality (XR) scene anchoring more accessible, versatile and robust than ever.

Read on to learn about ARY's latest breakthroughs — and what’s next!

ARY's progress during Sprint 3 of the CORTEX2 Programme

Q: How would you summarise ARY's latest developments during Sprint 3 of the CORTEX2 programme?

A: During Sprint 3, ARY had the chance to develop an Ultra-Wideband add-on for CORTEX2. This device allows any XR device with Bluetooth capabilities to be compatible with the previously developed system. Additionally, ARY has tested the limit of its system and showed an accuracy of 10 cm in anchoring an XR scene. 

Q: What milestones did ARY reach during Sprint 3, and what impact do they have?

A: In Sprint 3, ARY has developed an Ultra-Wideband tag that can be placed on the back of any smartphone. This tag makes any device virtually compatible with the previously developed anchoring system. Its accuracy is similar to smartphones already natively compatible with the system, allowing users to share the same experience.

Moreover, ARY has tested its solution in various conditions. The accuracy of the solution remained stable, no matter the distance from the beacons, their height or the devices used. Only people crossing the line of sight of the beacons could impact the results. However, these errors could be filtered out.

In conclusion, during the last nine months, ARY has developed a wireless and transparent XR anchoring system, based on its patented positioning algorithm. It achieved the expected accuracy of 10 centimetres in various conditions. Finally, the scope of compatible devices was extended thanks to an Ultra-Wideband Tag.

Q: What would you highlight about the Support Programme?

A: We would like to show our gratitude to the CORTEX2 consortium for making this project possible. It has been an enriching experience, and we are proud of what we have accomplished through this co-development.

A special thanks to our mentor, Bastian Krayer. His support over these past nine months has really helped us to conduct our project!

Check out ARY's previous interviews and stay updated on its progress!

Want to know more about other CORTEX2 innovators' updates? Browse all our supported teams on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: SENSO3D's 2nd progress update

Sprint 2 of the CORTEX2 Innovators Support Programme has moved the SENSO3D team forward, bringing significant advancements across AI capabilities, asset creation, and the development of immersive environments. The team is now focusing on conference room components and deploying AI-powered tools for rapid scene generation, which enhances accessibility and utility in virtual environments. From building a large-scale 3D model repository to delivering polished Unity scenes ready for real-world use, SENSO3D is laying a strong foundation for scalable, high-quality extended reality (XR) content.

Continue reading to learn more about the SENSO3D team's major milestones and next steps within the CORTEX2 programme.

SENSO3D's progress during Sprint 2 of the CORTEX2 programme

Q: How would you summarise SENSO3D's latest advancements within the CORTEX2 programme?

A: During Sprint 2 of the SENSO3D project, progress was made in various key areas:

• Expansion of the 3D model library: Over 100 new high-quality 3D models were added, including office furniture, audiovisual equipment, and decorative elements. These were developed using structured-light scanning, photogrammetry, and AI-driven reconstruction.
• Enhanced AI capabilities: Object detection algorithms were refined, enabling classification into 50 categories using the OMNI3D dataset and the F-Cube R-CNN model. New AI techniques significantly improved 2D-to-3D reconstruction, allowing for fast and accurate model generation from a single image​.
• Unity Integration & Scene Development: A streamlined import workflow was implemented, automating model retrieval, scaling, texture assignments, and collision detection. Two new immersive environments, a 3D Lobby and a 3D Conference Room, were developed for AR/VR applications​.
• Object Restoration & Optimisation: Mesh anomalies, texture misalignments, and polygon inconsistencies were resolved to ensure models met AR/VR standards. AI-enhanced model restoration techniques were applied to scanned assets​.
• Deviations & Adjustments: The project shifted its focus from household objects to conference rooms due to market demand. Additionally, commercially available models were integrated to accelerate progress, and a new Prompt-Based Scene Creation Tool was introduced to automate virtual environment generation.

Q: What key milestones has SENSO3D achieved, and what impact have they had?

A: Sprint 2 was a crucial phase for the SENSO3D project, bringing several key advancements in AI, 3D modelling, and integration workflows.

Here are the most significant milestones and their impact:

Expansion of the 3D model library

• Milestone: Over 100 new high-quality 3D models were added to the repository, including office chairs, podiums, AV equipment, and decorative elements. These models were developed using structured-light scanning, photogrammetry, and AI-driven reconstruction.
• Impact: Enriched the SENSO3D asset library with realistic, high-fidelity models tailored for AR/VR environments. Provided developers with ready-to-use, customisable assets for immersive virtual spaces.

AI enhancements

• Milestone:
  - AI-driven 3D object identification improved, allowing classification into 50 distinct categories.
  - Advanced 2D-to-3D reconstruction techniques (OMNI3D dataset, F-Cube R-CNN model) drastically enhanced accuracy and efficiency.
  - Introduced Sparse View 3D reconstruction, allowing high-quality 3D models to be created from just four images.
• Impact:
  - Faster and more accurate object detection and reconstruction reduce manual modelling efforts.
  - Enabled scalability, making SENSO3D more accessible for various industries requiring AR/VR-ready assets.
  - Positioned the project ahead of schedule in AI advancements, exceeding initial expectations.

Unity integration & scene development

• Milestone:
  - Developed a streamlined Unity import workflow, including:
  - Automated model retrieval and texture assignments.
  - Collision detection & prefab creation for AR/VR compatibility.
  - Designed two immersive environments:
  - 3D Lobby – A welcoming virtual entrance with interactive elements.
  - 3D Conference Room – A functional, realistic meeting space for AR/VR collaboration.
• Impact:
  - Simplified asset deployment for developers by ensuring seamless integration into Unity.
  - Created realistic, immersive environments for virtual meetings, exhibitions, and training sessions.
  - Strengthened the project's usability for real-world applications, particularly in business and education sectors.

Object restoration & quality standardisation

• Milestone:
  - Developed a rigorous AI-driven restoration process to fix:
  - Mesh gaps, texture misalignments, and polygon irregularities.
  - Improved UV mapping for accurate texture application.
  - Optimised polygon count to balance visual fidelity and AR/VR performance.
• Impact:
  - Ensured high-quality, optimised assets for resource-limited platforms like VR headsets & mobile AR applications.
  - Reduced manual intervention in model corrections, saving development time.

Introduction of the prompt-based scene creation tool

• Milestone:
  - Developed a prototype AI tool that allows users to create virtual scenes from text prompts.
  - Automates search, retrieval, and placement of 3D objects into Unity environments.
• Impact:
  - Revolutionises scene creation for non-technical users, making AR/VR design accessible.
  - Reduces manual work, accelerating content generation for XR experiences.

Strategic shift from household objects to conference room focus

• Milestone:
  - Based on market demand, the project pivoted from household objects to conference room components.
  - Integrated commercially available 3D models to accelerate progress.
• Impact:
  - Aligns with real-world use cases, enhancing project relevance and adoption.
  - Expedited development by combining scanned + commercial models instead of manually creating everything.

Q: What are SENSO3D's next steps within the CORTEX2 programme?

A: Our following main tasks are:

Finalising the 3D model repository

• Tasks:
  - Complete the categorisation, optimisation, and metadata tagging of all 3D assets.
  - Ensure all models meet predefined quality standards for geometry, textures, and Unity compatibility.
  - Expand the library with additional AI-generated models.
• Expected impact:
  - A fully structured and searchable 3D object repository ready for use in AR/VR applications.
  - Enhanced discoverability and reusability for developers working within the CORTEX2 framework.

Refining & polishing unity environments

• Tasks:
  - Finalise the 3D Lobby and Conference Room environments, ensuring high visual and functional quality.
  - Implement additional interactivity (e.g., dynamic object manipulation, adjustable furniture layouts).
  - Conduct end-to-end testing of the Unity import pipeline.
• Expected impact:
  - A seamless user experience in virtual environments, increasing adoption for business and training use cases.
  - Robust, high-performance Unity scenes for AR/VR applications.

Enhancing AI capabilities & scene creation tool

• Tasks:
  - Further optimise AI object detection & reconstruction for improved accuracy and speed.
  - Refine the Prompt-Based Scene Creation Tool, enabling users to generate entire 3D scenes from text descriptions.
  - Integrate NLP (Natural Language Processing) enhancements for better understanding of user input.
• Expected impact:
  - Faster, AI-assisted scene creation, reducing manual effort in virtual space design.
  - Broader accessibility, allowing even non-technical users to design immersive environments effortlessly.

Performance optimisation for AR/VR deployment

• Tasks:
  - Implement performance optimisation techniques such as:
  - Level-of-Detail (LOD) adjustments for scalability.
  - Texture compression to reduce load times.
  - Occlusion culling to improve rendering efficiency.
  - Conduct rigorous performance profiling across VR headsets, desktops, and mobile devices.
• Expected impact:
  - Optimised performance across different platforms, ensuring smooth interactions and high frame rates.
  - Scalability for enterprise use cases, making SENSO3D accessible to a broader audience.

Pilot testing & user feedback collection

• Tasks:
  - Deploy the 3D environments in pilot scenarios, including:
  - Virtual business meetings
  - Collaborative training sessions
  - Interactive exhibitions
  - Gather feedback from real users to refine usability and interaction design.
• Expected impact:
  - Real-world validation of SENSO3D's capabilities within the CORTEX2 ecosystem.
  - Iterative improvements based on user insights for better adoption.

Preparing for project handover & integration into CORTEX2

• Tasks:
  - Ensure full compatibility with CORTEX2's WebXR and WebGL frameworks.
  - Finalise technical documentation for developers who will integrate SENSO3D into their solutions.
  - Conduct a final review & testing phase before official project completion.
• Expected impact:
  - Seamless integration into the CORTEX2 ecosystem, allowing wider adoption of SENSO3D assets.
  - A well-documented, scalable, and ready-to-use 3D object library for XR applications.

Check out SENSO3D's previous interview and stay updated on its progress!

Want to know more about other CORTEX2 innovators' updates? Browse all our supported teams on the CORTEX2 website:

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: VISOR's 2nd progress update

The VISOR project has made significant strides in Sprint 2 of the CORTEX2 Innovators Support Programme, bringing its 3D reconstruction service closer to a fully functional and accessible solution. The team has integrated the service into a user-friendly platform, which marks a key milestone in making advanced technology accessible to a broader audience.

Keep reading to learn about the key achievements and future steps in VISOR.

VISOR's progress during Sprint 2 of the CORTEX2 programme

Q: How would you summarise VISOR's latest advancements within the CORTEX2 programme?

A: We have successfully integrated our 3D reconstruction service into a user-friendly website.

Q: What key milestones has VISOR achieved, and what impact have they had?

A: In Sprint 2, we have integrated our 3D reconstruction service into a website, enabling the easy uploading of images or videos in order for our service to 3D reconstruct the physical object and create its digital twin. We have also disseminated our work at the Stereopsia event in Brussels.

Q: What are VISOR's next steps within the CORTEX2 programme?

A: Our next steps are to further test our service with more input data (images or videos) of various objects. We will also generate documentation and guidelines on how to improve the reconstruction quality. Finally, a business plan will be generated describing the exploitation plan of the resulted outcome.

Check out VISOR's previous interview and stay updated on its progress!

Want to know more about other CORTEX2 innovators' updates? Browse all our supported teams on the CORTEX2 website:

Open Call 1 winners  -  Open Call 2 winners

by admin