admin

CORTEX2 innovators: XR-CARE's 1st progress update

Q: What is XR-CARE in one sentence?

A: XR-CARE is a modular, multimodal anonymisation framework designed to ensure privacy in XR teleconferencing by detecting and obfuscating sensitive information in video, audio, and text data.

Q: What problem are you solving? What makes your solution unique?

A: With XR-CARE, Logimade is addressing the growing privacy risks associated with recording and sharing XR teleconferencing sessions, where sensitive personal information—such as faces, voices, and on-screen text—can be unintentionally exposed. Current anonymisation tools are either limited to single data modalities or are too slow and complex for practical use in large-scale deployments.

What makes our solution unique is its modular, multimodal, and multi-stage architecture, which allows for configurable, high-recall anonymisation of video, audio, and text streams. It operates efficiently on consumer-grade hardware, supports real-time anonymisation, and adapts to different teleconferencing contexts (e.g., XR, desktop, mobile). Additionally, our system emphasises usability and transparency, allowing users to customise parameters, track processing history, and optimise anonymisation based on their needs—something no off-the-shelf solution currently offers.

Q: What are XR-CARE’s main objectives?

A: 

1. Develop a multimodal anonymisation framework capable of processing video, audio, and text from XR teleconferencing sessions while preserving contextual usability.
2. Ensure high privacy protection through a multi-stage detection strategy that minimises false negatives and adapts to varied teleconference scenarios.
3. Enable near real-time anonymisation using consumer-grade hardware, making the solution practical and scalable for widespread adoption.
4. Provide a user-friendly web platform where users can manage projects, customize anonymization parameters, and track processing history.
5. Integrate XR-CARE anonymisation platform with CORTEX2 framework.

CORTEX2 support programme progress

Q: What were the main activities implemented and milestones achieved during Sprint 1 of the CORTEX2 Support Programme?

A:

1. Development infrastructure was successfully established, including GPU-enabled environments, version control, and benchmarking tools to support reproducible research and AI-based processing.
2. Multi-scenario datasets were collected and annotated, capturing diverse XR teleconference conditions across video, audio, and text modalities, including challenging cases such as occlusions and varied lighting.
3. Comprehensive benchmarking of face and body detection algorithms was performed, evaluating models like YOLOv10 and MediaPipe for accuracy, speed, and robustness in realistic teleconferencing scenarios.
4. Initial evaluations of text and voice detection and obfuscation techniques were completed, with comparative tests of models such as FAST for text and Silero VAD for speech, along with analysis of obfuscation methods including Gaussian blur, pitch shifting, and spectral modification.
5. A modular software architecture was defined, enabling configurable, multi-stage anonymisation pipelines with support for adaptive processing and multimodal data integration.

Q: What have you achieved so far?

A: So far, XR-CARE has successfully progressed from the solution design and component evaluation (Sprint 1) to delivering an integrated, tested, and multi-modal anonymisation solution ready for deployment (Sprint 2).
During Sprint 2, three major milestones were achieved:

1. Integration with the CORTEX2 Platform: The team developed a production-ready RESTful API and a user-friendly web interface, enabling seamless access to and control of the anonymisation pipeline. This integration lays the groundwork for scalable, real-world application of the XR-CARE system within the CORTEX2 ecosystem.
2. Extension of Multi-Modal Anonymisation: The system was expanded to support visual anonymisation of health-related IoT sensor data embedded in video recordings, reinforcing the framework’s robustness in healthcare contexts and enhancing its capacity to process diverse XR data streams.
3. Comprehensive Testing and Debugging: Extensive testing was carried out using real-world XR teleconference datasets to validate performance, optimise speed and accuracy, and ensure compliance with GDPR. The resulting framework demonstrated high recall, low false positive rates, and efficient processing on consumer-grade hardware.

These developments have transformed XR-CARE into a practical, multimodal anonymisation solution capable of supporting privacy-preserving XR teleconferencing. The project has increased the team’s technological readiness and positioned XR-CARE for final validation and deployment in real-world CORTEX2 use cases.

Q: How is participating in CORTEX2 supporting XR-CARE?

A: The most valuable aspect of CORTEX2 support has been the technical mentorship provided by Alireza Javanmardi, whose guidance has been critical in refining our multi-stage anonymisation strategy. His input helped us make key architectural decisions, while the teleconference recordings from Open Rainbow he shared enabled more realistic and rigorous validation of our system.

Additionally, the encouragement to submit a full article and poster to EuroXR 2025, along with financial support for conference participation, has provided an excellent opportunity for dissemination, visibility, and networking. This not only helps to promote our work but also opens doors for potential collaborations and economic exploitation of the XR-CARE platform.

Q: What are your next steps within the CORTEX2 Programme?

A: The next steps of the project XR-CARE focus on bringing the framework to ready for public release through real-world testing, refinement, and dissemination:

1. Real-World Validation: We will conduct validation trials in real healthcare teleconferencing scenarios to assess the framework’s effectiveness. This phase will include the collection of performance metrics and user feedback to evaluate the system’s robustness, usability, and compliance with privacy standards.
2. Final Optimisation: Based on insights gathered during validation, we will implement targeted improvements to the anonymisation pipeline. Particular attention will be given to optimising detection performance, reducing processing time, and incorporating feedback from healthcare professionals and other end users.
3. Promotion and Dissemination: We will conduct a final review to ensure all project objectives have been fulfilled. In parallel, we will prepare promotional materials, including presentations, documentation, and online content, to support the visibility and potential adoption of XR-CARE beyond the CORTEX2 program.

Learn more about XR-CARE and stay updated on its progress!

Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: VISIXR's 1st progress update

Q: What is VISIXR in one sentence?

A: VISIXR aims to develop and deploy an innovative Smart Generator tool for real-time, AI-driven 3D asset modification within XR environments, aligning with the broader CORTEX2 vision of accessible and advanced immersive platforms.

Q: What problem are you solving? What makes your solution unique?

A: VISIXR provides a platform that breaks down 3D models into particular segments, analyses them and can modify them using voice or text input. At the same time, questions can be asked about the segments, which are then answered by the AI bot.

Q: What are VISIXR’s main objectives?

A: 

1. ¡Automatic Image Segmentation (2D und 3D)
2. Unity 3D Integration
3. Enhanced 3D asset modification in real-time
4. User-friendly UI

CORTEX2 support programme progress

Q: What were the main activities implemented and milestones achieved during Sprint 1 of the CORTEX2 Support Programme?

A: During Sprint 1 of the Support Programme, the main activities involved three core tasks. First, the team created foundational components for image segmentation and real-time modification, resulting in a prototype that could independently identify and act on image regions. Second, they integrated these tools with Unity3D, enabling real-time 3D rendering for use in extended reality (XR) environments. Finally, they conducted preliminary testing and refinement to optimise the prototype's performance, stability, and resource allocation.

The key milestones achieved were the successful development of a functional prototype capable of real-time segmentation, the successful integration with Unity3D for 3D manipulation, and the establishment of a robust system foundation for future sprints. Despite challenges with real-time processing and AI integration, all tasks were completed within the revised timeframe.

Q: What have you achieved so far?

A: In Sprint 2, the project made significant progress by extending the Smart Generator to interactive 3D applications and preparing its integration into the CORTEX2 framework. A key milestone was the extension to 3D assets, which means that users can now select and modify components of 3D models using voice and text input. New camera controls and enhanced visualisation methods, such as highlighting and animated exploded views, were developed for this purpose.

At the same time, the user interface was fundamentally revised based on user tests in order to make the operation more intuitive and significantly enhance the user experience. At the same time, the technical integration into the CORTEX2 environment was planned in detail, creating a clear roadmap for the future connection. In particular, the concept of "function calls" lays the foundation for being able to control the generator dynamically and contextually using external services.

To summarise, Sprint 2 has produced a much more interactive and immersive application, and at the same time set the decisive technical course for future integration and extended functionality.

Q: How is participating in CORTEX2 supporting VISIXR?

A: The CORTEX2 project has helped us in that it is a great environment to be part of a much larger project. In this way, you come into contact with other project groups and also see what others are currently working on, what ideas they have and accordingly what problems they are trying to solve. The dialogue within the project was also a great support. This took place either in the form of meetings with our mentor or in keynote sessions where approaches to various topics proposed by project groups were presented and discussed.

Q: What are your next steps within the CORTEX2 Programme?

A: The next steps in the program are to try to complete all outstanding tasks by the end of the 3rd sprint so that the generator can then be used. These include, for example, improving real-time modification and user-friendliness.

Learn more about VISIXR and stay updated on its progress!

Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: VIRTEX's 1st progress update

Q: What is VIRTEX in one sentence?

A: VIRTEX is a no-code XR training platform that empowers non-technical users to create and adapt immersive, multi-scenario simulations focused on decision-making in various industries.

Q: What problem are you solving? What makes your solution unique?

A: Traditional XR training solutions are costly, technically complex, and hard to adapt without specialised developers — making them inaccessible to many organisations and industries. VIRTEX addresses this by removing the technical barriers to XR content creation, enabling trainers, educators, and domain experts to build and adapt immersive simulations themselves through a no-code platform. This empowers sectors like healthcare, industry, education, and public services to scale decision-based training faster, cheaper, and more effectively.

What makes our solution unique is:

1. Truly no-code & user-friendly: VIRTEX empowers trainers, educators, and subject matter experts to build and customise immersive simulations without writing a single line of code.
2. Cross-industry flexibility: The platform supports multi-scenario simulations across diverse sectors — from healthcare and manufacturing to customer service and education.
3. Focus on decision-making: Scenarios are centred on realistic, branching decision points that reflect complex, real-world situations — not just technical skills.
4. Adaptable & modular: Content can be reused, localised, and adapted to different roles, contexts, or training levels — reducing both cost and time to deploy.
5. Rapid prototyping & deployment: Organisations can create or update training simulations in hours or days, supporting agility in workforce development.

https://www.youtube.com/watch?v=dLDDik3VMX0

Q: What are VIRTEX’s main objectives?

A:

• Develop a No-Code XR Editor: Create an intuitive, drag-and-drop platform that enables non-technical users to design and adapt immersive, interactive training scenarios without coding skills.
• Promote Cross-Industry Usability: Design the platform to be flexible and applicable across various sectors such as healthcare, manufacturing, education, customer service, and public services.
• Facilitate Rapid Adaptation: Allow users to quickly modify or build new scenarios to keep pace with evolving training needs, regulations, or organisational changes.
• Ensure Accessibility and Inclusivity: Make XR training accessible to users with varying technical backgrounds, languages, and learning styles to maximise impact.

CORTEX2 support programme progress

Q: What were the main activities implemented and milestones achieved during Sprint 1 of the CORTEX2 Support Programme?

A:

Main Activities Implemented

• MetaMedicsVR and LMU collaboratively developed a detailed project specification and test plan, outlining technical requirements, integration points with CORTEX2 SDKs, and testing methodologies.
• MetaMedicsVR team underwent training sessions to gain proficiency with CORTEX2 SDKs and APIs, exploring their capabilities relevant for virtual environment editing and integration.
• Development of the core functionalities of the virtual environment editor, including drag-and-drop UI design, backend architecture, and initial integration with CORTEX2 services.

Milestones Achieved

1. Completion of detailed Specification & Test Plan.
2. Proficiency gained with CORTEX2 SDKs and APIs.
3. Delivery of a prototype of the Virtual Environment Editor with basic service integration capabilities.

Q: What have you achieved so far?

A:

Main Activities Implemented

• Integration of at least five CORTEX2 services, such as scene synchronisation, Rainbow core, Cortex DB, Rainbow mediation gateway, gesture interaction, and security protocols, into the editor.
• Conducted rigorous compatibility testing to ensure the editor works seamlessly across multiple browsers and VR headsets.
• Performance tuning and backend infrastructure optimisation to support scalability and robust operation under load.
• Usability tests led by LMU gathered user feedback to refine and improve the UI for better intuitiveness and accessibility.
• Creation of comprehensive user guides in multiple languages and a demo video showcasing VIRTEX capabilities and usage.

Milestones Achieved

1. Successful integration of at least 5 CORTEX2 services.
2. Completion of compatibility testing across at least three platforms.
3. Backend optimisation for scalability finalised.
4. User interface refined based on usability testing feedback.
5. User documentation and demo video created

Q: How is participating in CORTEX2 supporting VIRTEX?

A: Participating in the CORTEX2 Support Programme has significantly accelerated the development and deployment of VIRTEX. Through access to advanced XR technologies and the CORTEX2 SDKs, we have integrated cutting-edge features such as real-time scene synchronisation and gesture interaction, which would have been challenging and costly to develop independently. The expert mentoring and technical support provided by the consortium have helped us address complex challenges efficiently, enhancing the platform’s quality and robustness. Additionally, the program has facilitated valuable networking opportunities, enabling collaboration with other innovators and expanding our project’s impact. Being part of this prestigious EU initiative has also increased our visibility and credibility, which is crucial for engaging stakeholders and potential customers. Furthermore, access to diverse user groups through CORTEX2 has allowed us to gather essential feedback, ensuring VIRTEX meets real-world needs across various industries. Overall, CORTEX2 has been a catalyst that boosts our technical progress, market readiness, and ecosystem integration.

Q: What are your next steps within the CORTEX2 Programme?

A: Our next steps within the CORTEX2 program focus on finishing comprehensive testing of the VIRTEX platform to ensure stability and usability across target environments. We will document all findings and learnings gathered during development and user trials to guide future improvements. Concurrently, we will develop strategies to scale VIRTEX effectively, including expanding industry applications, enhancing platform robustness, and exploring partnerships to broaden adoption. These actions aim to maximise VIRTEX’s impact and readiness for real-world deployment.

Learn more about VIRTEX and stay updated on its progress!

Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: VEM's 1st progress update

Q: What is VEM in one sentence?

A: Our primary long-term objective is to deliver an easily implementable and accessible tool that will assist developers in integrating end-to-end encryption (E2E) into VR projects.

Q: What problem are you solving? What makes your solution unique?

A: We address the lack of VR-specific, privacy-by-design solutions for secure communication in professional and research environments. Our SDK uniquely integrates end-to-end encryption directly into VR platforms (Unity3D), ensuring maximum data privacy and ease of implementation.

Q: What are VEM’s main objectives?

A: The main goal is to enable secure, end-to-end encrypted communication within XR applications. We focus on delivering a developer-friendly SDK and API that substantially increases privacy on VR.

CORTEX2 support programme progress

Q: What were the main activities implemented and milestones achieved during Sprint 1 of the CORTEX2 Support Programme?

A: During Sprint 1, we built the core architecture and ensured system stability. This foundational work minimised future risks and set the stage for advanced features in later phases. We also developed smart key distribution.

Q: What have you achieved so far?

A: In Sprint 2, we implemented PrivMX technology (e2e libraries) in Unity3D and next integrated with the Rainbow platform, enhancing VR communication privacy significantly.

Q: How is participating in CORTEX2 supporting VEM?

A: CORTEX2 gave us access to experts genuinely interested in applying our technology, which led to the development of a realistic and relevant use case. We also received valuable support in integrating our solution into a broader system, allowing us to validate its interoperability. Additionally, the opportunity to explore other breakthrough technologies within the program has inspired new ideas and accelerated the professionalisation of our approach to VR.

Q: What are your next steps within the CORTEX2 Programme?

A: We plan to continue testing and validation while preparing for broader integration and deployment. Our goal is to collaborate with developers and stakeholders to refine the SDK and expand its adoption in real-world VR applications.

Learn more about VEM and stay updated on its progress!

Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: SAME-XR's 1st progress update

Q: What is SAME-XR in one sentence?

A: A VCS-Integrated Asset Management Pipeline to Bridge the Gap Between 3D Artists and XR Developers

Q: What problem are you solving? What makes your solution unique?

A: Recent industry surveys reveal that most studios spend more than an hour per week on basic asset management tasks like searching for the latest files (53%) and transforming formats (62%). The same data reveals a clear demand for specific solutions, with 77% of studios rating version control as a very or extremely important feature in an asset management system. This insight became the cornerstone of our approach in our project SAME-XR (Scalable Asset Management and Conversion Engine for XR), a platform designed to solve these inefficiencies.

Q: What are SAME-XR’s main objectives?

A: Depending on the model complexity, less than 5-10 minutes of processing time of new models (Including conversions, preview generation, meta data, etc.).
Minimum 3-4 supported formats including .obj .gltf/.glb
Handling of at least 15 simultaneous requests.

CORTEX2 support programme progress

Q: What were the main activities implemented and milestones achieved during Sprint 1 of the CORTEX2 Support Programme?

A: The sprint focused on implementing the core features of the SAME-XR platform, covering four key tasks: (1) backend API development, (2) file operations services, (3) web interface creation, and (4) additional interface development. The system is designed as a microservices-based, event-driven, cloud-native SaaS application, leveraging technologies such as PostgreSQL, AWS S3, AWS SNS/SQS, and Keycloak for authentication and storage.

A proof-of-concept (PoC) version of the system was successfully deployed on an AWS EC2 instance for testing and validation. Key performance indicators (KPIs) were met, demonstrating high efficiency in model processing, broad format compatibility, and scalable request handling. No deviations were encountered from the planned development roadmap

Q: What have you achieved so far?

A: Sprint 2 involved a strategic pivot from the initial goal of direct CORTEX2 integration towards developing standalone tools, API clients, and integration with version control systems (VCS), specifically GitLab, to better align with established developer workflows.

Development focused on creating the foundational “Projects microservice” to manage assets, issues, and VCS integration; functional prototypes of a Blender add-on and a Unity package for direct user interaction; a suite of supporting API clients (Python, C#, JavaScript, Unity); and comprehensive Helm charts for Kubernetes deployment. Key achievements include an operational API for the Projects microservice, a successful proof-of-concept demonstrating the automated GitLab issue synchronisation workflow, and functional Blender/Unity prototypes enabling core asset upload and download.

Q: How is participating in CORTEX2 supporting SAME-XR?

A: The SAME-XR project has benefited greatly from involvement in the CORTEX² program. The most beneficial element has been our incorporation into the thriving CORTEX² ecosystem, which goes beyond the necessary funding. Our strategy shift to a more reliable, developer-centric solution has been directly influenced by the valuable insights we have gained from working with top partners, interacting with a variety of use cases, and receiving focused mentoring. Being a part of this network has greatly expanded our perspective on the prospects and real-world issues within the European XR ecosystem, in addition to speeding up our technical development.

Q: What are your next steps within the CORTEX2 Programme?

A: Our immediate next steps within the CORTEX² program are centred on rigorous testing and user-centred refinement. Now that the core functionalities of the SAME-XR pipeline and its integrated add-ons are in place, our priority is to deploy the solution for hands-on testing with the CORTEX² partners. We will focus on gathering detailed qualitative and quantitative feedback on the workflow's usability, performance, and real-world impact. This crucial feedback loop will drive our final phase of development, allowing us to polish the user interfaces, resolve any remaining bottlenecks, and ensure SAME-XR delivers maximum value to the ecosystem.

Learn more about SAME-XR and stay updated on its progress!

Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: R3in3D's 1st progress update

Q: What is R3in3D in one sentence?

A: R3in3D delivers real-time, photorealistic 3D representations of humans and objects to revolutionise remote training and collaboration in XR environments.

Q: What problem are you solving? What makes your solution unique?

A: We address the growing need for immersive, high-fidelity remote training and collaboration tools in industrial and educational settings. Traditional XR systems lack realism, responsiveness, and human presence. R3in3D introduces a modular holographic communication pipeline that supports multiple 3D representation formats — RGBD, stereo, and 2D chroma — enabling seamless, lifelike interaction over next-gen networks (5G/6G). Our solution uniquely blends real-time volumetric capture, Unity-based rendering, and WebRTC transmission for scalable, low-latency XR experiences.

Q: What are R3in3D’s main objectives?

A:

• Enable immersive, real-time remote training using photorealistic 3D representations.
• Integrate RGBD and stereo capture formats into the CORTEX2 platform.
• Optimise communication efficiency and rendering quality for XR applications.
• Validate the system through real-world use cases in technical training and remote collaboration.

https://www.youtube.com/watch?v=5XFPV2Zzosg

CORTEX2 support programme progress

Q: What were the main activities implemented and milestones achieved during Sprint 1 of the CORTEX2 Support Programme?

A:

• Completed benchmarking and integration planning with the CORTEX2 platform.
• Designed the R3in3D architecture, including Capture Module, Unity Client, and Signalling Server.
• Developed a Unity-based client for transmitting and rendering volumetric video using Rainbow SDK.
• Defined KPIs and test plans for use case validation.

Q: What have you achieved so far?

A:

• Integrated RGBD and stereo 3D formats into the CORTEX2 pipeline.
• Developed a fully functional Unity client with real-time audio/video/metadata streaming.
• Demonstrated the system at IMX’25 and ICT.Open, receiving strong industry interest and a Best Demo nomination.
• Published and submitted multiple peer-reviewed papers, advancing the state of XR-based remote training.

Q: How is participating in CORTEX2 supporting R3in3D?

A: CORTEX2 has been instrumental in accelerating our development through access to cutting-edge SDKs, expert mentorship, and a collaborative innovation ecosystem. The program enabled rapid prototyping, real-world testing, and visibility across the European XR community.

Q: What are your next steps within the CORTEX2 Programme?

A: 

• Finalise rendering integration and support for all three representation formats.
• Launch user studies to evaluate training effectiveness and system usability.
• Prepare for broader deployment and commercialisation.

We’re excited to be shaping the future of XR-based training and collaboration. R3in3D is not just a prototype — it’s a platform for scalable, immersive learning and communication across industries.

Learn more about R3in3D and stay updated on its progress!

Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: PETER's 1st progress update

Q: What is PETER in one sentence?

A: Preserve Emotions in Translations for Extended Reality.

Q: What problem are you solving? What makes your solution unique?

A: PETER addresses the challenge of real-time voice translation in XR (Extended Reality) environments, aiming to overcome language barriers and preserve emotional tone in communication. This is especially relevant for industries like construction, maintenance, and remote assistance, where miscommunication can be critical.

What makes it unique:

• Targets quasi-real-time translation with ≤ 3 seconds latency.
• Preserves emotional tone (non-verbal cues) during translation.
• Uses a modular toolchain combining open-source components for ASR, emotion detection, translation, and TTS.
• Designed for integration with the CORTEX2 framework and released as open source.

Q: What are PETER’s main objectives?

A:

• Develop a voice-to-voice translation prototype that preserves emotional tone for XR.
• Achieve end-to-end latency ≤ 3 seconds.
• Ensure high accuracy in both translation and emotion preservation.
• Enable integration with CORTEX2 as a server-side service.
• Validate with CORTEX2 pilots using technical and subjective KPIs.
• Release the prototype as open source for community and CORTEX2 reuse.
• Enhance communication and reduce language barriers in XR environments.

https://youtu.be/HPUsP719nuw

CORTEX2 support programme progress

Q: What were the main activities implemented and milestones achieved during Sprint 1 of the CORTEX2 Support Programme?

A:

• Designed and built the voice-to-voice toolchain; selected, tuned, and benchmarked building blocks; toolchain design document, experimental software toolchain.
• Defined the interface between PETER and the CORTEX2 system (based on CoVA); interface documentation.
• Toolchain completed
• CORTEX2 interface completed
• Deployed the toolchain and integrated it live with the CORTEX2 platform; PETER running on a live CORTEX2 environment
• Performed fine-tuning and optimisation to meet KPIs; Updated design documentation
• Final integrated toolchain
• PETER MVP completed and operational

Impact:

• Provided a working prototype capable of real-time voice-to-voice translation with emotional tone preservation.
• Demonstrated feasibility and integration into the CORTEX2 ecosystem, laying the groundwork for user testing and real-world validation.

Q: How is participating in CORTEX2 supporting PETER?

A:

• Helped ensure that PETER fits smoothly into the CORTEX2 architecture, especially through reference to the CoVA component for speech interfaces.
• CORTEX2 provides strong channels for dissemination (events, website, other XR projects), boosting PETER’s exposure.
• As PETER enhances CORTEX2, the success of CORTEX2 in the market will also increase the visibility and adoption of PETER.

Q: What are your next steps within the CORTEX2 Programme?

A:

Task: Technical and Psychometric Validation
Objective: Validate PETER in a relevant environment, both technically and from the user perspective.
Activities:

• Run objective benchmarks to measure key performance indicators (e.g., latency, accuracy, emotion preservation).
• Engage CORTEX2 pilot users in subjective testing to assess:
  
• User satisfaction
  
• Perceived quality of emotional tone preservation
  
• Overall usability
  

Task: Final Review and Release
Objective: Consolidate results, define areas for improvement, and make the solution publicly available.
Activities:

• Conduct a final review of all validation outcomes.
  
• Define a plan for future improvements based on feedback.
  
• Release PETER as open-source software on GitHub, including full documentation.

Learn more about PETER and stay updated on its progress!

Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: NODAV's 1st progress update

Q: What is NODAV in one sentence?

A: Efficient compression and visualisation tools for 3D Gaussian Splatting to enable real-time rendering of lightweight 3D scenes across VR, web, and mobile platforms

Q: What problem are you solving? What makes your solution unique?

A: 3D Gaussian Splatting (3DGS) offers high-quality real-time 3D scene rendering, but the resulting datasets are large and computationally intensive—making them impractical for storage, transmission, and rendering on low-end or mobile devices. NODAV combines advanced pruning, quantisation, and real-time encoding/decoding techniques specifically tailored to 3DGS data, and integrates them into a unified pipeline that works seamlessly in Unity across VR, desktop, and web platforms—delivering high visual quality with drastically reduced file sizes (target: <50MB) and minimal performance trade-offs.

Q: What are NODAV’s main objectives?

A:

1. Reduce the size of 3D Gaussian Splatting (.ply and .splat) files through pruning and quantisation, aiming for lightweight 3D scene representations.
   
2. Develop an efficient real-time compression framework to encode/decode 3DGS data, enabling scalable cloud storage and streaming
3. Integrate the compressed 3DGS data into the Unity engine for cross-platform visualization on VR headsets, mobile devices, and desktop.
4. Seamlessly embed all modules into the CORTEX2 framework, including data synchronisation, scene storage, and client-side rendering capabilities.
5. Conduct technical validation and user testing to ensure visual quality, usability across devices, and user validation.
6. Share outcomes through scientific publications, public demos, dataset releases, and social media engagement to maximise impact and transparency.

CORTEX2 support programme progress

Q: What were the main activities implemented and milestones achieved during Sprint 1 of the CORTEX2 Support Programme?

A

Main activities:

1. Developed an automatic evaluation tool to measure the quality of 3D Gaussian Splatting (3DGS) reconstructions.
2. Built a pipeline using 360° cameras for efficient and high-quality scene capture.
3. Created a dataset of 15 museum rooms for testing and validation.
4. Applied data compression using pruning and spherical harmonics to reduce file size by up to 10×.
5. Designed an encoding method using video compression (H.265 preferred over H.264 for quality).
6. Developed a cross-platform renderer using Unity and Vulkan for Windows and Android VR devices.

Key milestones:

1. Quality evaluation system running with key image metrics.
2. Compression pipeline achieved up to 10× data reduction.
3. First version of encoding/decoding module completed.
4. Renderer deployed successfully on Windows and Meta Quest 3 (Android VR).
5. Achieved 100 FPS on Windows VR; 15–20 FPS on Android VR.
6. Dataset successfully recorded and processed (more than 10 scenes).

Q: What have you achieved so far?

A:

Main activities:

1. Introduced quantisation techniques to enhance the existing compression pipeline. Achieved up to 25× data reduction with minimal quality loss.
2. Switched to lossless LZW compression for critical attributes to maintain visual quality while reducing size.
3. Built a modular, API-based pipeline integrating reconstruction, compression, quantisation, and encoding, deployable via Docker.
4. Implemented single-pass multiview rendering to improve VR performance.
5. Integrated WebGPU rendering for browser-based visualisation.
6. Focused on untethered VR (Quest 3) with gains in rendering efficiency and stability.

Key milestones:

1. Compression pipeline achieved up to 25× reduction (goal: 30×).
2. New lossless encoder fully implemented and tested.
3. Cross-platform rendering extended to Web (WebGPU) and optimised for VR (single-pass).
4. Created a containerised service exposing each pipeline component via REST API.
5. Achieved >50 FPS in WebGPU, >100 FPS on desktop, and improved Quest 3 performance.
6. KPIs for compression and dataset recording were fulfilled; rendering speed and quality are ongoing.

Q: How is participating in CORTEX2 supporting NODAV?

A: CORTEX2 has directly supported our technical progress by providing a platform to validate and optimise our 3D Gaussian Splatting pipeline in real-world XR environments. It enabled us to test rendering performance on untethered VR devices, refine our compression pipeline, and integrate our system with CORTEX2’s web-based and mobile platforms.

Q: What are your next steps within the CORTEX2 Programme?

A: Our next steps can be summarised as:

- Full deployment and validation of our reconstruction, pruning and compression pipeline within CORTEX2 ecosystem.
- Set of pilots/demos to showcase to other partners and potential customers the technology we have developed.

Learn more about NODAV and stay updated on its progress!

Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: NITRUS's 1st progress update

Q: What is NITRUS in one sentence?

A: NITRUS is an advanced AI gesture recognition system that helps people interact with machines and virtual environments.

Q: What problem are you solving? What makes your solution unique?

A: Gesture recognition pipelines usually require a lot of computing resources and effort to make them work with a limited set of gestures. Our End-to-End AI pipeline processes video instead of single frames, working in real time and constrained resources with high precision.

Q: What are NITRUS’s main objectives?

A:

• To create a solution that enriches the interaction of people’s digital experiences by interpreting and identifying human gestures.
• To gather and/or create gesture datasets with variability and inclusion considerations that can be used for the development and validation of said solution.
• To develop an action recognition AI model based on Spatio-temporal CNNs capable of identifying people's actions in real-time.
• To provide tools for the optimisation and deployment of this model in hardware-constrained scenarios while maintaining real-time and bandwidth requirements.

CORTEX2 support programme progress

Q: What were the main activities implemented and milestones achieved during Sprint 1 of the CORTEX2 Support Programme?

A: In sprint 1, we have defined the requirements and specifications of our solution as well as familiarised ourselves with CORTEX, the state of the art and the available datasets. We have implemented and tested ~ 10 different AI models that fulfil one or more of the steps needed for the gesture recognition pipeline, covering two completely different approaches that may be used for the application.

Q: What have you achieved so far?

A: The current prototype of the NITRUS solution fulfils all the original expectations. The technical objectives and KPIs have been achieved, and the project is on track for the final stage of the implementation.
The current prototype is able to detect 14 different gestures with 92% accuracy. It can be executed in CPU and GPU, making it compatible with resource-constrained devices, and it works with a ~270ms latency and ~ 2s response time.

Q: How is participating in CORTEX2 supporting NITRUS?

A: CORTEX2 has supported us in the development of the technology and provided a way to integrate our technology into teleconference applications.

Our mentors have given us interesting feedback from both the technology and use case perspectives, helping us to create a solution that is modular and adaptable to many scenarios.

Q: What are your next steps within the CORTEX2 Programme?

A: Fully validate the technology and exploit the results in other gesture recognition applications.

Learn more about NITRUS and stay updated on its progress!

Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: INTERACT's 1st progress update

Q: What is INTERACT in one sentence?

A: INTERACT is an AI-driven project that develops real-time sign language translation through 3D avatars in Extended Reality (XR), enabling deaf, hard-of-hearing, and multilingual participants to fully engage in business meetings and teleconferences by integrating speech-to-text, sentiment analysis, and collaborative tools within an inclusive, scalable environment built on the CORTEX2 architecture

Q: What problem are you solving? What makes your solution unique?

A: INTERACT addresses the pressing issue of exclusion in remote professional communication by enabling deaf, hard-of-hearing, and multilingual individuals to fully participate in business meetings and teleconferences, where current systems often lack real-time sign language translation, emotional context awareness, and inclusive collaborative tools. What makes INTERACT unique is its integration of AI-powered 3D avatars capable of performing real-time sign language translation within immersive XR environments using Meta Quest 3 headsets, enhanced by sentiment analysis that allows avatars to reflect emotional cues and foster empathetic interactions. Unlike traditional solutions, INTERACT supports multilingual speech-to-text and text-to-sign translation in International Sign Language (ISL), while leveraging the CORTEX2 architecture, including the Rainbow SDK and Mediation Gateway. The system also provides added functionalities such as automatic meeting summarisation, live subtitling, and information overlays, all optimised for low-bandwidth and sustainable operation. This convergence of accessibility, emotional intelligence, multilingual support, and immersive interaction positions INTERACT as a pioneering and holistic solution for inclusive digital communication.

Q: What are INTERACT’s main objectives?

A: The key objectives of INTERACT are to develop AI-powered 3D avatars for real-time sign language translation within immersive AR/XR environments, enhance multilingual communication through integrated speech-to-text and text-to-sign translation in International Sign Language, and enable empathetic interactions via sentiment analysis. The project aims to ensure accessibility and inclusivity in teleconferencing by leveraging the CORTEX2 architecture. Additionally, it focuses on validating the platform in real-world business and educational settings, collecting user feedback to refine avatar behaviour, and promoting broader adoption through strategic dissemination and stakeholder engagement.

CORTEX2 support programme progress

Q: What were the main activities implemented and milestones achieved during Sprint 1 of the CORTEX2 Support Programme?

A: During Sprint 1, INTERACT focused on defining technical specifications, developing a detailed test plan, and establishing a robust data management framework aligned with FAIR principles. Key activities included configuring the Whisper AI speech-to-text model, exploring AI models for sentiment analysis and sign language translation, and integrating initial components of the CORTEX2 architecture. The team also created the project website and launched dissemination activities via LinkedIn. The main milestone achieved was the delivery of Deliverable D1: Specification and Test Plan, laying the groundwork for integration and development in the following sprints.

Q: What have you achieved so far?

A: During Sprint 2, INTERACT successfully integrated core CORTEX2 components, such as the Rainbow SDK, Mediation Gateway, and sentiment-enhanced avatars, into an immersive XR environment, enabling real-time speech-to-text transcription and initial sign language translation features. ISL models were built using online datasets, and sign animations were extracted using AI models, with avatar animations and emotional expressions tested for realistic interactions. A proof-of-concept prototype and demonstration video were delivered as key milestones, showcasing the system's functionality and accessibility impact. These achievements marked a major step forward in validating the platform's technical feasibility and readiness for user testing in real-world scenarios.

Q: How is participating in CORTEX2 supporting INTERACT?

A: Participating in CORTEX2 has provided our team with essential funding, access to advanced XR technologies, and technical support through the CORTEX2 architecture, enabling us to accelerate development, validate our solution in real-world scenarios, and scale an inclusive communication platform for deaf, hard-of-hearing, and multilingual users.

Q: What are your next steps within the CORTEX2 Programme?

A: Our next steps within the CORTEX2 program include conducting pilot demonstrations involving deaf participants, gathering user feedback through questionnaires, refining avatar animations and multilingual translation features based on real-world interactions, and validating our final KPIs through comprehensive user testing to ensure the platform’s accessibility, usability, and impact.

Learn more about INTERACT and stay updated on its progress!

Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website: 

Open Call 1 winners  -  Open Call 2 winners

by admin

CORTEX2 innovators: CORE-MHC's 2nd progress update

Throughout the CORTEX2 Support Programme, CORE-MHC has explored how immersive technologies can be applied to remote mental health care, with promising results. By developing and testing a multiuser Mixed Reality platform designed to support therapeutic interactions at a distance, the team has shown that immersive environments and co-designed experiences can meaningfully enhance both patient engagement and clinical effectiveness. In this final phase, CORE-MHC validated their solution with mental health professionals and users, refined their approach, and laid the groundwork for broader adoption in the healthcare sector.

In the interview below, the CORE-MHC team reflect on what they’ve achieved, how the programme has supported their growth, and what comes next.

CORE-MHC's progress on the CORTEX2 Programme

Q: How would you summarise the advances CORE-MHC has made during Phase 2 of the CORTEX2 Support Programme?

A: During Phase 2 of the project, significant progress was achieved with the successful completion of two key milestones:

• Development of a Minimum Viable Product (MVP)/Proof of Concept (PoC): Development of a multiuser 3D gamified scenario to support remote therapeutic activities in Mixed Reality (MR). Two application configurations were implemented: one allowing therapists to use a PC while connecting with a patient using a Mixed Reality headset, and another where both patient and therapist wear Mixed Reality headsets. In the first modality, the PC application allows the therapist to observe what the patient is seeing through the MR headset in real time, as well as display the patient's biofeedback captured from wearable sensors such as a smartwatch and a respiration band. In the second modality, both users connect to the same digital environment from two different physical locations; they can see each other’s avatars and interact with each other and with the environment in real time.
• Preliminary validation with end users: The PoC was evaluated by therapists at Fundación SASM, who provided valuable feedback that allowed us to refine the technological solution before the validation phase.

Q: What has CORE-MHC achieved now that the Programme is complete?

A: During Phase 3, we conducted a user study with both mental health professionals and users of mental health services at Fundación SASM. ITI defined the experimental design and prepared visual materials and tutorials as well as questionnaires focused on presence, usability and user experience. The results yielded very positive outcomes towards the use of MR in remote therapy for mental healthcare.

Users generally found the application helpful for their specific problems and were satisfied with their therapist's treatment. Users of mental health services preferred the headset-to-headset modality, while professionals preferred the PC-to-headset modality due to familiarity and the possibility of taking notes simultaneously.

We have identified several design guidelines for developing Mixed Reality experiences for mental health that provide a solid foundation for future developers willing to leverage MR for remote mental health care.

The CORE-MHC project has demonstrated the transformative potential of MR technologies in delivering accessible, engaging, and effective remote mental health care. By integrating immersive environments with real-time physiological data and co-designed therapeutic experiences, CORE-MHC offers a novel approach to improving patient motivation, adherence, and outcomes.

Q: What would you highlight about the Support Programme, what's helped advance your solution the most?

A: The CORTEX2 Support Programme has played a vital role in the advancement of our solution, offering both technical and collaborative benefits that have significantly accelerated our development process. The integration of CORTEX2 components — especially the IoT server and Rainbow SDK — helped us build a robust and interoperable Mixed Reality environment. These tools enabled secure real-time communication and data visualisation between therapist and patient, enhancing therapeutic interaction.

The collaboration between ITI and Fundación SASM has been instrumental in grounding the project in real-world therapeutic needs and ensuring that the platform is both clinically relevant and user-friendly.

Q: What's the status of CORE-MHC after completing the Programme? What are your next steps?

A: As next steps, we will explore other aspects of health that remote immersive scenarios, such as physiotherapy, cognitive training, emotional regulation, or group therapy sessions, could facilitate. We will also disseminate our work to other entities in the mental health sector, such as hospitals, NGOs, private clinics, public institutions, and other kinds of entities which could benefit from incorporating novel tools to support mental health.

Check out CORE-MHC's previous interview and stay updated on its progress!

Want to know more about other CORTEX2 innovators' updates? Browse all our supported teams on the CORTEX2 website:

Open Call 1 winners  -  Open Call 2 winners

by admin