Q: What is PETER in one sentence?
A: Preserve Emotions in Translations for Extended Reality.
Q: What problem are you solving? What makes your solution unique?
A: PETER addresses the challenge of real-time voice translation in XR (Extended Reality) environments, aiming to overcome language barriers and preserve emotional tone in communication. This is especially relevant for industries like construction, maintenance, and remote assistance, where miscommunication can be critical.
What makes it unique:
- Targets quasi-real-time translation with ≤ 3 seconds latency.
- Preserves emotional tone (non-verbal cues) during translation.
- Uses a modular toolchain combining open-source components for ASR, emotion detection, translation, and TTS.
- Designed for integration with the CORTEX2 framework and released as open source.
Q: What are PETER’s main objectives?
A:
- Develop a voice-to-voice translation prototype that preserves emotional tone for XR.
- Achieve end-to-end latency ≤ 3 seconds.
- Ensure high accuracy in both translation and emotion preservation.
- Enable integration with CORTEX2 as a server-side service.
- Validate with CORTEX2 pilots using technical and subjective KPIs.
- Release the prototype as open source for community and CORTEX2 reuse.
- Enhance communication and reduce language barriers in XR environments.
CORTEX2 support programme progress
Q: What were the main activities implemented and milestones achieved during Sprint 1 of the CORTEX2 Support Programme?
A:
- Designed and built the voice-to-voice toolchain; selected, tuned, and benchmarked building blocks; toolchain design document, experimental software toolchain.
- Defined the interface between PETER and the CORTEX2 system (based on CoVA); interface documentation.
- Toolchain completed
- CORTEX2 interface completed
- Deployed the toolchain and integrated it live with the CORTEX2 platform; PETER running on a live CORTEX2 environment
- Performed fine-tuning and optimisation to meet KPIs; Updated design documentation
- Final integrated toolchain
- PETER MVP completed and operational
Impact:
- Provided a working prototype capable of real-time voice-to-voice translation with emotional tone preservation.
- Demonstrated feasibility and integration into the CORTEX2 ecosystem, laying the groundwork for user testing and real-world validation.
Q: How is participating in CORTEX2 supporting PETER?
A:
- Helped ensure that PETER fits smoothly into the CORTEX2 architecture, especially through reference to the CoVA component for speech interfaces.
- CORTEX2 provides strong channels for dissemination (events, website, other XR projects), boosting PETER’s exposure.
- As PETER enhances CORTEX2, the success of CORTEX2 in the market will also increase the visibility and adoption of PETER.
Q: What are your next steps within the CORTEX2 Programme?
A:
Task: Technical and Psychometric Validation
Objective: Validate PETER in a relevant environment, both technically and from the user perspective.
Activities:
- Run objective benchmarks to measure key performance indicators (e.g., latency, accuracy, emotion preservation).
- Engage CORTEX2 pilot users in subjective testing to assess:
- User satisfaction
- Perceived quality of emotional tone preservation
- Overall usability
Task: Final Review and Release
Objective: Consolidate results, define areas for improvement, and make the solution publicly available.
Activities:
- Conduct a final review of all validation outcomes.
- Define a plan for future improvements based on feedback.
- Release PETER as open-source software on GitHub, including full documentation.
Learn more about PETER and stay updated on its progress!
Want to explore more XR innovation? Browse all our supported projects on the CORTEX2 website: