Citation: | Chen Tianshi, Tie Yun, Qi Lin, et al. An improved method to render the sound of VR scene[J]. Opto-Electronic Engineering, 2018, 45(6): 170742. doi: 10.12086/oee.2018.170742 |
[1] | Tsingos N, Gallo E, Drettakis G. Perceptual audio rendering of complex virtual environments[J]. ACM Transactions on Graphics, 2003, 23(3): 249-258. |
[2] | Moeck T, Bonneel N, Tsingos N, et al. Progressive perceptual audio rendering of complex scenes[C]//Proceedings of the 2007 Symposium on Interactive 3D Graphics and Games, 2007: 189-196. |
[3] | Schissler C, Manocha D. Interactive sound propagation and rendering for large multi-source scenes[J]. ACM Transactions on Graphics, 2016, 36(4): 121-139 |
[4] | 鲁溟峰, 倪国强, 白廷柱, 等.基于分数阶傅里叶变换的量化噪声抑制方法[J].北京理工大学学报, 2015, 35(12): 1285-1290. Lu M F, Ni G Q, Bai T Z, et al. A novel method for suppressing the quantization noise based on fractional Fourier transform[J]. Transactions of Beijing Institute of Technology, 2015, 35(12): 1285-1290. |
[5] | ITU. Methods for the subjective assessment of small impairments in audio systems including multichannel sound systems: ITU-R BS. 1116-1[R]. Geneva: ITU, 1994: 1128-1136. |
[6] | Hochbaum D S, Shmoys D B. A best possible heuristic for the k-center problem[J]. Mathematics of Operations Research, 1985, 10(2): 180-184. doi: 10.1287/moor.10.2.180 |
[7] | Schissler C, Nicholls A, Mehra R. Efficient HRTF-based spatial audio for area and volumetric sources[J]. IEEE Transactions on Visualization and Computer Graphics, 2016, 22(4): 1356-1366. doi: 10.1109/TVCG.2016.2518134 |
[8] | Takala T, Hahn J. Sound rendering[J]. ACM SIGGRAPH Computer Graphics, 1992, 26(2): 211-220. doi: 10.1145/142920 |
[9] | Schissler C, Loftin C, Manocha D. Acoustic classification and optimization for multi-modal rendering of real-world scenes[J]. IEEE Transactions on Visualization and Computer Graphics, 2017, 24(3): 1246-1259. |
[10] | Taylor M T, Chandak A, Antani L, et al. RESound: interactive sound rendering for dynamic virtual environments[C]//Proceedings of the 17th ACM International Conference on Multimedia, 2009: 271-280. |
[11] | Raghuvanshi N, Snyder J, Mehra R, et al. Precomputed wave simulation for real-time sound propagation of dynamic sources in complex scenes[J]. ACM Transactions on Graphics (TOG), 2010, 29(4): 142-149. |
[12] | Grelaud D, Bonneel N, Wimmer M, et al. Efficient and practical audio-visual rendering for games using crossmodal perception[C]//Proceedings of the 2009 Symposium on Interactive 3D Graphics and Games, 2009: 177-182. |
[13] | Innami S, Kasai H. On-demand soundscape generation using spatial audio mixing[C]//Proceedings of 2011 IEEE International Conference on Consumer Electronics, 2011: 29-30. |
Overview: In the field of VR, spatial sound rendering technology plays an increasingly important role. The spatial sense of sound plays a very important role in enhancing the immersive sense of the user in the VR scene. At present, the advanced space sound rendering scheme is mainly divided into two types: the sound waveform and the ray-based tracking algorithm. Recently, important research includes the following points: Tsingos proposed a dynamic spatial sound rendering method based on culling and clustering, which made it possible to process large-scale sound sources in real time. Moeck considered the visual factors during clustering, which reduced the clustering cost. The new algorithm proposed by Schissler eliminated the impact of obstacles on the clustering results. Tao et al. proposed quantization noise suppression method based on fractional Fourier transform to improve the quality of the audio signal sampling. Based on the virtual scene containing hundreds of movable sound sources, due to the high computational cost of clustering stage, the traditional spatial sound rendering scheme often takes up too much computing resources, which has become a bottleneck in the development of VR audio rendering technology. In this paper, we improve the processing speed of sound rendering and the operation efficiency of the entire system by adding the average angle deviation threshold in the clustering step. In addition, we design and implement a perceptual user experiment that validates the notion that people are more susceptible to spatial errors in different types of sound sources, especially if it is visible. Based on this conclusion, this paper proposes an improved method of sound clustering, which reduces the possibility of different types of sound sources clustering. Summarized as follows: focusing on rendering of complex virtual auditory scenes comprising hundreds of moving sound sources using spatial audio mixing, we propose a new clustering algorithm considering average angle error. We presented an effectiveness of our algorithm over specific condition to reduce computational costs caused by frequently clustering. In addition, the result of subjective experiments expresses that the clustering of different types of sound sources will cause more spatial information errors. Using this result, this paper proposes a method based on sound source label parameters, which solves the problem of clustering different kinds of sound sources. In the end, three scene experiments verified the feasibility of the new method.
The process of sound rendering program
The influence of two conditions on sound rendering
The results of two clustering algorithms. (a) Traditional algorithm; (b) Improved algorithm