Networked microcontrollers for accessible, distributed spatial audio DOI Creative Commons

Thomas Albert Rushton,

Romain Michon,

Stefania Serafin

et al.

Frontiers in Virtual Reality, Journal Year: 2024, Volume and Issue: 5

Published: Nov. 8, 2024

State-of-the-art systems for spatial and immersive audio are typically very costly, being reliant on specialist hardware capable of performing computationally intensive signal processing delivering output to many tens, if not hundreds, loudspeakers. Centralised this sort suffer from limited accessibility due their inflexibility expense. Building the research past few decades in transmission data over computer networks, emergence recent years increasingly capable, low-cost microcontroller-based development platforms with support both networking functionality, we present a prototype decentralised, modular alternative. Having previously explored feasibility running microcontroller device as networked client, here describe client-server system improved scalability via multicast transmission. The operates ubiquitous, commonplace computing equipment, view it simple, versatile, highly-accessible platform, granting users freedom explore spatialisation approaches at vastly reduced Though faced by significant technical challenges, particularly regard maintaining synchronicity between distributed processors, produces perceptually plausible results. Findings commensurate capability, further research, disrupt democratise fields audio.

Language: Английский

5G-Enabled Internet of Musical Things Architectures for Remote Immersive Musical Practices DOI Creative Commons
Luca Turchet, Claudia Rinaldi, Carlo Centofanti

et al.

IEEE Open Journal of the Communications Society, Journal Year: 2024, Volume and Issue: 5, P. 4691 - 4709

Published: Jan. 1, 2024

Networked Music Performances (NMPs) involve geographically-displaced musicians performing together in real-time. To date, scarce research has been conducted on how to integrate NMP systems with immersive audio rendering techniques able enrich the musicians' perception of sharing same acoustic environment. In addition, use wireless technologies for NMPs largely overlooked. this paper, we propose two architectures Immersive (INMPs), which differ physical positions computing blocks constituting 3D toolchain. These leverage a backend specifically conceived support remote musical practices via Software Defined Networking methods, and takes advantage orchestration, slicing, Multi-access Edge Computing (MEC) capabilities 5G. Moreover, illustrate machine learning algorithms network traffic prediction packet loss concealment. Traffic predictions at multiple time scales are utilized achieve an optimized placement Virtual Network Functions hosting mixing processing functionalities within available MEC sites, depending users' geographical locations current load conditions. An analysis technical requirements INMPs using is provided, along their performance assessment simulators.

Language: Английский

Citations

12
Edge-Enabled Spatial Audio Service: Implementation and Performance Analysis on a MEC 5G Infrastructure DOI
Federico Martusciello, Carlo Centofanti, Claudia Rinaldi

et al.

Published: Oct. 26, 2023

Spatial audio technologies are becoming a fundamental requirement for guaranteeing immersive auditory experiences in various applications such as Augmented and Virtual Reality up to the Metaverse. With rise of mobile edge computing, there is growing interest exploring spatial algorithms performance on infrastructures. This paper presents an evaluation two different potential offloading real time processing Mobile Edge Computing (MEC) infrastructure. The presented results were obtained through evaluations performed operator network, they demonstrate feasibility computation at network edge. No difference terms between was observed under assumed scenario.

Language: Английский

Citations

5
Investigation of Server–Based Spatial Audio for Metaverse Concert Distribution DOI
Patrick Cairns, Helena Daffern, Gavin Kearney

et al.

Published: Sept. 30, 2024

Modern music delivery contexts include the real-time distribution of musical performances as immersive media ('metaverse concerts') to online audiences. This paper details an investigatory deployment a novel server-based spatial audio rendering system for this context, considering practical wireless Virtual Reality (VR) audience endpoint. Ambisonic auralisation is provided with 6 Degrees Freedom (6DoF) using Spatial Impulse Response (SIR) interpolation approach which efficient large-scale distribution. Binaural distributed mobile device at endpoint low-latency Network Music Performance (NMP) streaming software. Improvement in achieved relative that possible on consumer HMDs. Latency measurements suggest acceptable motion-sound latency not device, however may be achievable other contexts, or optimised systems.

Language: Английский

Citations

0
Locally Adapted Immersive Environments for Distributed Music Performances in Mixed Reality DOI
Andrea Genovese,

Marta Gospodarek,

Zack Nguyen

et al.

Published: Sept. 30, 2024

The integration of network-based music performance with immersive media technology opens new compelling avenues for live collaborative multimedia concerts and exhibitions. This paper presents a development workflow creating cohesive environments network performances, where an exhibition node is connected to remote ensemble. These rely on the usage local characterization data, such as room acoustic measurements digital twin assets, render mix audio motion-capture streams, capable merging performers into single shared display streamed eXtended Reality devices. Spatialization auralization techniques are used add realism auditory elements ground them in acoustics by musicians stage audiences. resulting "hybrid" combines virtual-reality mixed-reality principles create specific type novel interactive concert experience.

Language: Английский

Citations

0
Telemersive Audio Systems for Online Jamming DOI
Patrick Müller,

Roman Haefeli,

Johannes Schütt

et al.

Published: Sept. 30, 2024

This paper focuses on aesthetic approaches, specific projects and tools for online jamming involving immersive audio. By 'online jamming', we refer to the widespread practice of individual performers located in different spaces collaborating via Internet near real-time. 'immersive audio', mean capture, processing delivery sound with three-dimensional directional characteristics — specifically, through binaural rendering headphones. The first part spatial dimension telematic performance practices, which, compared temporal said is under-conceptualised relevant literature. second highlights some perceptual requirements that need be considered when musicians play together interact each other rendering. We then analyse a number tools, technical applications artistic can subsumed under an expanded understanding audio, two which were developed our own research unit. analysis namely draws how these aesthetically deal categories they respond challenges. concludes by measuring latencies audio plug-ins streaming utilities. results show potential such telemersive approaches promising designed address latency issues, are critical jamming.

Language: Английский

Citations

0
Networked microcontrollers for accessible, distributed spatial audio DOI Creative Commons

Thomas Albert Rushton,

Romain Michon,

Stefania Serafin

et al.

Frontiers in Virtual Reality, Journal Year: 2024, Volume and Issue: 5

Published: Nov. 8, 2024

State-of-the-art systems for spatial and immersive audio are typically very costly, being reliant on specialist hardware capable of performing computationally intensive signal processing delivering output to many tens, if not hundreds, loudspeakers. Centralised this sort suffer from limited accessibility due their inflexibility expense. Building the research past few decades in transmission data over computer networks, emergence recent years increasingly capable, low-cost microcontroller-based development platforms with support both networking functionality, we present a prototype decentralised, modular alternative. Having previously explored feasibility running microcontroller device as networked client, here describe client-server system improved scalability via multicast transmission. The operates ubiquitous, commonplace computing equipment, view it simple, versatile, highly-accessible platform, granting users freedom explore spatialisation approaches at vastly reduced Though faced by significant technical challenges, particularly regard maintaining synchronicity between distributed processors, produces perceptually plausible results. Findings commensurate capability, further research, disrupt democratise fields audio.

Language: Английский

Citations

0