THE CORTEX2OPEN CALL #1 WINNERS

The RAX project takes inspiration from its namesake (the Alps mountainous range) to set a high but realistic ambition for developing a scalable, automatic, integrated tool for realistic, customisable, interoperable, and multimodal Avatars. It will be integrated with the CORTEX2 technological ecosystem to extend its capabilities by covering User Representation and User Avatar Customization.

FLYTEX aims to revolutionise decision-making in the agricultural sector by providing real-time IoT sensor data during videoconferences. Focusing on an industry that is rapidly adopting digital technologies, the project aims to gather and share expert insights efficiently. This innovative approach not only aligns with the trend of digitalisation in agriculture but also introduces a new dimension of strategic decision-making by enabling data-driven insights. By integrating IoT data into communication platforms, FLYTEX enhances the quality, speed, and efficacy of decision-making processes, making it a vital tool in the modern agricultural landscape.

SenseGlove is embarking on the development of a multiplayer toolkit, designed to empower XR developers in seamlessly creating interactive virtual environments featuring haptic gloves and hand tracking. Their strategy involves harnessing the capabilities of the CORTEX2 framework, specialising in the management of multiplayer objects, avatars, and scenes.

Complementing this toolkit, SenseGlove will provide end-users with a template scene comprising 10 pre-fabricated interactable assets. This serves as an entry point, enabling non-developers to effortlessly construct multiplayer interactive scenarios, ensuring smooth interoperability between hand tracking and haptic (and force-feedback) gloves.

SENSO3D is dedicated to building a comprehensive 3D object library meticulously curated for home appliances and household components. But what sets SENSO3D apart is its innovative AI-powered model selector tool, designed to identify elements in 2D photos and seamlessly replace them with appropriate 3D models from an extensive database. This breakthrough technology promises to transform 2D images into enriched 3D scenes, unlocking a world of possibilities across various domains.

The project envisions creating detailed and accurate 3D models for XR applications, with a focus on areas such as elder care, language learning, and interactive education. By converting 2D images into immersive 3D environments, SENSO3D enhances visualisation and interaction, offering substantial benefits to users, including those with special needs.

Efficient 3D reconstruction is a key operation that can transform the content generation process of many applications, such as mixed reality (MR) applications, movies, game development, telepresence, 3D printing and 3D eCommerce. VISOR proposes a web service that will take images or a video stream of a small object and generate a digital twin as a triangular mesh that can be used by all current XR applications and game engines and be visualised on any device, enabling easy sharing of the 3D model across multiple stakeholders and environments.

EXTERNALISE will develop a disruptive multi-user collaboration and communication module that will change how teams and groups collaborate remotely. By focusing on digitising and streaming the human characteristics that encode the nonverbal cues that compose human body language, EXTERNALISE will enrich users’ representation and boost their expressivity during communication.

ARY is an augmented reality (AR) media which offers the capability to anchor 3D objects, videos, pictures or PDF files into indoor environments and make them available to anyone using a smartphone or other device.

This project aims to develop and integrate the Magos Gestures Library (MGL) module into the CORTEX2 framework, optimising the interactions landscape of extended reality (XR) applications. Its significance lies in the transformative potential it holds for almost all kinds of XR applications, such as simulations, collaborative training, and realistic interactions in industry applications such as Healthcare, Aerospace, and industry 5.0.

MGL, an extension of the innovative Magos solution, focuses on advancing hand tracking and gesture recognition technologies, with a specific emphasis on personalized and dynamic gestures, thanks to its highly accurate tracking system. The Magos Gestures Library will enable users to define and recognise specific gestures — a minimum of 15— tailored to distinct actions within XR environments.

TIP aims to enrich the CORTEX2 platform with an innovative 2D/3D asset library optimised for Unity and Mozilla Hubs. Focusing on education, entertainment and culture sectors, they plan to create immersive and adaptable learning environments, including a traditional classroom, a group study room, and a library for individual learning, alongside interactive spaces for virtual concerts and cultural exhibitions. These environments, comprising a blend of static and dynamic assets, will be customizable to user needs. With these versatile, high-quality 3D assets, they aim to foster engaging educational experiences and enhance the cultural and entertainment appeal of the CORTEX2 platform, potentially broadening its user base and community engagement.

The development of a dynamic library of personalised gestures is essential in enhancing the user experience in augmented reality (AR) and virtual reality (VR) applications. CDLPG will create a module capable of accurately capturing and interpreting a wide range of hand gestures that can be used across various AR/VR applications within the CORTEX2 platform. This technology will make interactions more intuitive and will facilitate greater adoption of AR/VR technology in different sectors.

AgriVision integrates XR with farm management information systems (FMIS), revolutionising how farmers can interpret complex data through intuitive, immersive visualisations. This innovation not only facilitates quicker and more efficient decision-making but also introduces a paradigm shift in farmer-consultant interactions, enabling real-time immersive XR-visits and remote collaboration. This significantly diminishes the need for frequent in-person consultations, enhancing overall farming operational efficiency.

Agrivision will be offered in two versions: a “lite version” aiming to run on farmers mobile devices and a “pro version” aiming to fully utilise XR capabilities, utilising dedicated XR devices, such as Ηololens. The project will involve agricultural experts/consultants and farmers for pilot testing and feedback collection.

The project will design, create, and test a cutting-edge virtual reality simulation environment tailored for use in hospital settings to support rehabilitation. Its goal is to provide a versatile tool available across all existing VR platforms, offering immersive or semi-immersive experiences with various modes of interaction. It will enhance patient involvement, enable remote accessibility, and contribute to a more comprehensive understanding of disease progression.

• SCIPLANT is an innovative XR-based application designed to revolutionise urban planning. It integrates immersive technologies with real-time data to create dynamic urban models, enhancing efficiency and accuracy in city planning.

Aimed at fostering sustainable development, this tool facilitates collaborative decision-making among urban planners, architects, government officials, and citizens. Its gender-neutral design and customisable features ensure accessibility and inclusivity, accommodating a diverse range of user needs. SCIPLANT is a testament to innovation in digital urban planning, paving the way for future advancements in sustainable city design.

The HYMNE use case in CORTEX2 aims to open a new angle on hybrid music events. It will focus on gathering the largest audience over time (rather than at the same time) by creating unique interactive immersive concerts in which the audience plays an important part.

3-time Grammy winner and guitarist Steve Vai believes that hybrid concerts will further revolutionise the music industry. In 2026, the project aims to be able to organise full hybrid events: live shows augmented with an XR experience. After the event, the audience can book an interactive VR concert. Artists will perform both in the live show and in the VR concert, volumetrically recorded or live-streamed. Steve Vai will provide feedback on the virtual stage performance during the project.

• XRisis will create inclusive, engaging and easily repeatable simulated virtual crisis environments. It will implement and pilot an MVP for collaborative emergency and crisis management training and upskilling.

Three exemplar crisis management pilots will be built on top of CORTEX2 & its services and tested by a minimum of 30 crisis response personnel remotely across a minimum of four countries, which will provide an evidence-based validation that a real time XR communications environment can improve collaborative learning experiences, increase adoption, drive costs, increase training delivery efficiencies, and decrease logistical complexities.

The C.A.R.E. XR project aims to revolutionise emergency management by integrating XR with the Next-Generation Incident Command System (NICS). This initiative will enhance situational awareness and decision-making through real-time, 3D XR visualizations, and IoT data integration. Leveraging machine learning and natural language processing, the project will improve communication efficiency between first responders and command centres.

Aligning with the CORTEX² framework, it is set to establish a new standard in emergency response, reduce operational times, and increase safety for both responders and civilians. This initiative represents a major step towards the future of technologically advanced emergency services.

CORE-MHC aims to define the next generation of XR experiences for mental healthcare. It will develop a gamified platform to support remote and co-facilitated mental health treatment using mixed reality. By making therapy engaging, interactive, and accessible, it will address the treatment gap in mental healthcare, enhance healthcare accessibility, and positively impact the European healthcare landscape.

FocusVR: ADHD VR Solutions integrates cutting-edge VR technology to revolutionize ADHD management. It aims to design immersive VR scenarios specifically tailored to improve cognitive skills in ADHD patients, including attention, memory, and emotional regulation. This innovative approach addresses the need for engaging, customised cognitive training, aligning with CORTEX2’s healthcare vision. The project promises significant advancements in ADHD therapy, offering a novel, effective tool for patients and clinicians alike, and setting new standards in VR-based healthcare solutions.

SELFEX2 aims to improve training processes in manufacturing by using wearable finger tracking gloves and XR. It will allow for a synchronous self-training approach that provides a quantifiable degree of readiness to execute a dexterity-based task in the workplace.

By integrating the SELFEX2 concept into the CORTEX2 framework, the real-time training between a teacher-senior operator at one location and several junior operators in remote locations, who can learn to execute dexterity-based tasks by combining video, voice, and the XR representation of the hands of the teacher, will be possible.

VR technology can provide a paradigm shift in education with high-quality, inclusive, and sustainable tools using real-life examples. The vScientist project will develop a comprehensive XR & VR platform for enhancing the learning and accessibility of Computational Fluid Dynamics (CFD) for students and individuals from diverse backgrounds, promoting social inclusion and accessibility in STEM fields. Through this novel platform, users will visualise, interact, and analyse 3D virtual experiments in an immersive environment or run their own in seconds using machine learning.