Currently,the main idea of iterative rendering methods is to allocate a fixed number of samples to pixels that have not been fully rendered by calculating the completion rate.It is obvious that this strategy ignores t...Currently,the main idea of iterative rendering methods is to allocate a fixed number of samples to pixels that have not been fully rendered by calculating the completion rate.It is obvious that this strategy ignores the changes in pixel values during the previous rendering process,which may result in additional iterative operations.展开更多
With technological advancements,virtual reality(VR),once limited to high-end professional applications,is rapidly expanding into entertainment and broader consumer domains.However,the inherent contradiction between mo...With technological advancements,virtual reality(VR),once limited to high-end professional applications,is rapidly expanding into entertainment and broader consumer domains.However,the inherent contradiction between mobile hardware computing power and the demand for high-resolution,high-refresh-rate rendering has intensified,leading to critical bottlenecks,including frame latency and power overload,which constrain large-scale applications of VR systems.This study systematically analyzes four key technologies for efficient VR rendering:(1)foveated rendering,which dynamically reduces rendering precision in peripheral regions based on the physiological characteristics of the human visual system(HVS),thereby significantly decreasing graphics computation load;(2)stereo rendering,optimized through consistent stereo rendering acceleration algorithms;(3)cloud rendering,utilizing object-based decomposition and illumination-based decomposition for distributed resource scheduling;and(4)low-power rendering,integrating parameter-optimized rendering,super-resolution technology,and frame-generation technology to enhance mobile energy efficiency.Through a systematic review of the core principles and optimization approaches of these technologies,this study establishes research benchmarks for developing efficient VR systems that achieve high fidelity and low latency while providing further theoretical support for the engineering implementation and industrial advancement of VR rendering technologies.展开更多
Background In recent years,the demand for interactive photorealistic three-dimensional(3D)environments has increased in various fields,including architecture,engineering,and entertainment.However,achieving a balance b...Background In recent years,the demand for interactive photorealistic three-dimensional(3D)environments has increased in various fields,including architecture,engineering,and entertainment.However,achieving a balance between the quality and efficiency of high-performance 3D applications and virtual reality(VR)remains challenging.Methods This study addresses this issue by revisiting and extending view interpolation for image-based rendering(IBR),which enables the exploration of spacious open environments in 3D and VR.Therefore,we introduce multimorphing,a novel rendering method based on the spatial data structure of 2D image patches,called the image graph.Using this approach,novel views can be rendered with up to six degrees of freedom using only a sparse set of views.The rendering process does not require 3D reconstruction of the geometry or per-pixel depth information,and all relevant data for the output are extracted from the local morphing cells of the image graph.The detection of parallax image regions during preprocessing reduces rendering artifacts by extrapolating image patches from adjacent cells in real-time.In addition,a GPU-based solution was presented to resolve exposure inconsistencies within a dataset,enabling seamless transitions of brightness when moving between areas with varying light intensities.Results Experiments on multiple real-world and synthetic scenes demonstrate that the presented method achieves high"VR-compatible"frame rates,even on mid-range and legacy hardware,respectively.While achieving adequate visual quality even for sparse datasets,it outperforms other IBR and current neural rendering approaches.Conclusions Using the correspondence-based decomposition of input images into morphing cells of 2D image patches,multidimensional image morphing provides high-performance novel view generation,supporting open 3D and VR environments.Nevertheless,the handling of morphing artifacts in the parallax image regions remains a topic for future research.展开更多
High-fidelity tactile rendering offers significant potential for improving the richness and immersion of touchscreen interactions.This study focuses on a quantitative description of tactile rendering fidelity using a ...High-fidelity tactile rendering offers significant potential for improving the richness and immersion of touchscreen interactions.This study focuses on a quantitative description of tactile rendering fidelity using a custom-designed hybrid electrovibration and mechanical vibration(HEM)device.An electrovibration and mechanical vibration(EMV)algorithm that renders 3D gratings with different physical heights was proposed and shown to achieve 81%accuracy in shape recognition.Models of tactile rendering fidelity were established based on the evaluation of the height discrimination threshold,and the psychophysical-physical relationships between the discrimination and reference heights were well described by a modification of Weber’s law,with correlation coefficients higher than 0.9.The physiological-physical relationship between the pulse firing rate and the physical stimulation voltage was modeled using the Izhikevich spiking model with a logarithmic relationship.展开更多
Background Physics-based differentiable rendering(PBDR)aims to propagate gradients from scene parameters to image pixels or vice versa.The physically correct gradients obtained can be used in various applications,incl...Background Physics-based differentiable rendering(PBDR)aims to propagate gradients from scene parameters to image pixels or vice versa.The physically correct gradients obtained can be used in various applications,including inverse rendering and machine learning.Currently,two categories of methods are prevalent in the PBDR community:reparameterization and boundary sampling methods.The state-of-the-art boundary sampling methods rely on a guiding structure to calculate the gradients efficiently.They utilize the rays generated in traditional path-tracing methods and project them onto the object silhouette boundary to initialize the guiding structure.Methods In this study,we propose an augmentation of previous projective-sampling-based boundary-sampling methods in a bidirectional manner.Specifically,we utilize the rays spawned from the sensors and also employ the rays emitted by the emitters to initialize the guiding structure.Results To demonstrate the benefits of our technique,we perform a comparative analysis of differentiable rendering and inverse rendering performance.We utilize a range of synthetic scene examples and evaluate our method against state-of-the-art projective-sampling-based differentiable rendering methods.Conclusions The experiments show that our method achieves lower variance gradients in the forward differentiable rendering process and better geometry reconstruction quality in the inverse-rendering results.展开更多
The utilization of phosphors that achieve full-spectrum lighting has emerged as a prevailing trend in the advancement of white light-emitting diode(WLED)lighting.In this study,we successfully prepared a novel green ph...The utilization of phosphors that achieve full-spectrum lighting has emerged as a prevailing trend in the advancement of white light-emitting diode(WLED)lighting.In this study,we successfully prepared a novel green phosphor Ba_(2)Sc_(2)((BO_(3))_(2)B_(2)O_(5)):Ce^(3+)(BSBO:Ce^(3+))that can be utilized for full-spectrum lighting and low-temperature sensors.BSBO:Ce^(3+)exhibits a broad-band excitation spectrum centered at 410 nm,and a broad-band emission spectrum centered at 525 nm.The internal and external quantum efficiencies of BSBO:Ce^(3+)are 99%and 49%,respectively.The thermal stability of BSBO:Ce^(3+)can be improved by substituting partial Sc atoms with smaller cations.The thermal quenching mechanism of BSBO:Ce^(3+)and the lattice occupancy of Ce ions in BSBO are discussed in detail.Furthermore,by combining the green phosphor BSBO:Ce^(3+),the commercial blue phosphor and the red phosphor on a 405 nm chip,a white light source was obtained with a high average color rendering index(CRI)of 96.6,a low correlated color temperature(CCT)of 3988 K,and a high luminous efficacy of 88.0 Im/W.The lu-minous efficacy of the WLED exhibits negligible degradation during the 1000 h light aging experiment.What's more,an emission peak at 468 nm appears when excited at 352 nm and 80 K,however,the relative intensity of the peaks at 468 and 525 nm gradually weakens with increasing temperature,indicating the potential of this material as a low-temperature sensor.展开更多
Ray casting algorithm can obtain a better quality image in volume rendering, however, it exists some problems, such as powerful computing capacity and slow rendering speed. How to improve the re-sampled speed is a key...Ray casting algorithm can obtain a better quality image in volume rendering, however, it exists some problems, such as powerful computing capacity and slow rendering speed. How to improve the re-sampled speed is a key to speed up the ray casting algorithm. An algorithm is introduced to reduce matrix computation by matrix transformation characteristics of re-sampling points in a two coordinate system. The projection of 3-D datasets on image plane is adopted to reduce the number of rays. Utilizing boundary box technique avoids the sampling in empty voxel. By extending the Bresenham algorithm to three dimensions, each re-sampling point is calculated. Experimental results show that a two to three-fold improvement in rendering speed using the optimized algorithm, and the similar image quality to traditional algorithm can be achieved. The optimized algorithm can produce the required quality images, thus reducing the total operations and speeding up the volume rendering.展开更多
As a cornerstone for applications such as autonomous driving,3D urban perception is a burgeoning field of study.Enhancing the performance and robustness of these perception systems is crucial for ensuring the safety o...As a cornerstone for applications such as autonomous driving,3D urban perception is a burgeoning field of study.Enhancing the performance and robustness of these perception systems is crucial for ensuring the safety of next-generation autonomous vehicles.In this work,we introduce a novel neural scene representation called Street Detection Gaussians(SDGs),which redefines urban 3D perception through an integrated architecture unifying reconstruction and detection.At its core lies the dynamic Gaussian representation,where time-conditioned parameterization enables simultaneous modeling of static environments and dynamic objects through physically constrained Gaussian evolution.The framework’s radar-enhanced perception module learns cross-modal correlations between sparse radardata anddense visual features,resulting ina22%reduction inocclusionerrors compared tovisiononly systems.A breakthrough differentiable rendering pipeline back-propagates semantic detection losses throughout the entire 3D reconstruction process,enabling the optimization of both geometric and semantic fidelity.Evaluated on the Waymo Open Dataset and the KITTI Dataset,the system achieves real-time performance(135 Frames Per Second(FPS)),photorealistic quality(Peak Signal-to-Noise Ratio(PSNR)34.9 dB),and state-of-the-art detection accuracy(78.1%Mean Average Precision(mAP)),demonstrating a 3.8×end-to-end improvement over existing hybrid approaches while enabling seamless integration with autonomous driving stacks.展开更多
This paper presents a new solution to haptic based teleoperation to control a large-sized slave robot for space exploration, which includes two specially designed haptic joysticks, a hybrid master-slave motion mapping...This paper presents a new solution to haptic based teleoperation to control a large-sized slave robot for space exploration, which includes two specially designed haptic joysticks, a hybrid master-slave motion mapping method, and a haptic feedback model rendering the operating resistance and the interactive feedback on the slave side. Two devices using the 3 R and DELTA mechanisms respectively are developed to be manipulated to control the position and orientation of a large-sized slave robot by using both of a user's two hands respectively. The hybrid motion mapping method combines rate control and variable scaled position mapping to realize accurate and efficient master-slave control. Haptic feedback for these two mapping modes is designed with emphasis on ergonomics to improve the immersion of haptic based teleoperation. A stiffness estimation method is used to calculate the contact stiffness on the slave side and play the contact force rendered by using a traditional spring-damping model to a user on the master side stably. Experiments by using virtual environments to simulate the slave side are conducted to validate the effectiveness and efficiency of the proposed solution.展开更多
A new algorithm is proposed for restoring disocclusion regions in depth-image-based rendering （DIBR） warped images. Current solutions include layered depth image （LDI）, pre-filtering methods, and post-processing m...A new algorithm is proposed for restoring disocclusion regions in depth-image-based rendering （DIBR） warped images. Current solutions include layered depth image （LDI）, pre-filtering methods, and post-processing methods. The LDI is complicated, and pre-filtering of depth images causes noticeable geometrical distortions in cases of large baseline warping. This paper presents a depth-aided inpainting method which inherits merits from Criminisi＇s inpainting algorithm. The proposed method features incorporation of a depth cue into texture estimation. The algorithm efficiently handles depth ambiguity by penalizing larger Lagrange multipliers of flling points closer to the warping position compared with the surrounding existing points. We perform morphological operations on depth images to accelerate the algorithm convergence, and adopt a luma-first strategy to adapt to various color sampling formats. Experiments on test multi-view sequence showed that our method has superiority in depth differentiation and geometrical loyalty in the restoration of warped images. Also, peak signal-to-noise ratio （PSNR） statistics on non-hole regions and whole image comparisons both compare favorably to those obtained by state of the art techniques.展开更多
基金supported partially by the National Natural Science Foundation of China(No.U19A2063)the Jilin Provincial Science&Technology Development Program of China(No.20230201080GX)。
文摘Currently,the main idea of iterative rendering methods is to allocate a fixed number of samples to pixels that have not been fully rendered by calculating the completion rate.It is obvious that this strategy ignores the changes in pixel values during the previous rendering process,which may result in additional iterative operations.
基金Supported by the National Key R&D Program of China under grant No.2022YFB3303203the National Natural Science Foundation of China under grant No.62272275.
文摘With technological advancements,virtual reality(VR),once limited to high-end professional applications,is rapidly expanding into entertainment and broader consumer domains.However,the inherent contradiction between mobile hardware computing power and the demand for high-resolution,high-refresh-rate rendering has intensified,leading to critical bottlenecks,including frame latency and power overload,which constrain large-scale applications of VR systems.This study systematically analyzes four key technologies for efficient VR rendering:(1)foveated rendering,which dynamically reduces rendering precision in peripheral regions based on the physiological characteristics of the human visual system(HVS),thereby significantly decreasing graphics computation load;(2)stereo rendering,optimized through consistent stereo rendering acceleration algorithms;(3)cloud rendering,utilizing object-based decomposition and illumination-based decomposition for distributed resource scheduling;and(4)low-power rendering,integrating parameter-optimized rendering,super-resolution technology,and frame-generation technology to enhance mobile energy efficiency.Through a systematic review of the core principles and optimization approaches of these technologies,this study establishes research benchmarks for developing efficient VR systems that achieve high fidelity and low latency while providing further theoretical support for the engineering implementation and industrial advancement of VR rendering technologies.
基金Supported by the Bavarian Academic Forum(BayWISS),as a part of the joint academic partnership digitalization program.
文摘Background In recent years,the demand for interactive photorealistic three-dimensional(3D)environments has increased in various fields,including architecture,engineering,and entertainment.However,achieving a balance between the quality and efficiency of high-performance 3D applications and virtual reality(VR)remains challenging.Methods This study addresses this issue by revisiting and extending view interpolation for image-based rendering(IBR),which enables the exploration of spacious open environments in 3D and VR.Therefore,we introduce multimorphing,a novel rendering method based on the spatial data structure of 2D image patches,called the image graph.Using this approach,novel views can be rendered with up to six degrees of freedom using only a sparse set of views.The rendering process does not require 3D reconstruction of the geometry or per-pixel depth information,and all relevant data for the output are extracted from the local morphing cells of the image graph.The detection of parallax image regions during preprocessing reduces rendering artifacts by extrapolating image patches from adjacent cells in real-time.In addition,a GPU-based solution was presented to resolve exposure inconsistencies within a dataset,enabling seamless transitions of brightness when moving between areas with varying light intensities.Results Experiments on multiple real-world and synthetic scenes demonstrate that the presented method achieves high"VR-compatible"frame rates,even on mid-range and legacy hardware,respectively.While achieving adequate visual quality even for sparse datasets,it outperforms other IBR and current neural rendering approaches.Conclusions Using the correspondence-based decomposition of input images into morphing cells of 2D image patches,multidimensional image morphing provides high-performance novel view generation,supporting open 3D and VR environments.Nevertheless,the handling of morphing artifacts in the parallax image regions remains a topic for future research.
基金Supported by the National Natural Science Foundation of China under Grants 61631010 and 61806085.
文摘High-fidelity tactile rendering offers significant potential for improving the richness and immersion of touchscreen interactions.This study focuses on a quantitative description of tactile rendering fidelity using a custom-designed hybrid electrovibration and mechanical vibration(HEM)device.An electrovibration and mechanical vibration(EMV)algorithm that renders 3D gratings with different physical heights was proposed and shown to achieve 81%accuracy in shape recognition.Models of tactile rendering fidelity were established based on the evaluation of the height discrimination threshold,and the psychophysical-physical relationships between the discrimination and reference heights were well described by a modification of Weber’s law,with correlation coefficients higher than 0.9.The physiological-physical relationship between the pulse firing rate and the physical stimulation voltage was modeled using the Izhikevich spiking model with a logarithmic relationship.
基金Supported by National Natural Science Foundation of China(No.62072020)the Leading Talents in Innovation and Entrepreneurship of Qingdao,China(19-3-2-21-zhc).
文摘Background Physics-based differentiable rendering(PBDR)aims to propagate gradients from scene parameters to image pixels or vice versa.The physically correct gradients obtained can be used in various applications,including inverse rendering and machine learning.Currently,two categories of methods are prevalent in the PBDR community:reparameterization and boundary sampling methods.The state-of-the-art boundary sampling methods rely on a guiding structure to calculate the gradients efficiently.They utilize the rays generated in traditional path-tracing methods and project them onto the object silhouette boundary to initialize the guiding structure.Methods In this study,we propose an augmentation of previous projective-sampling-based boundary-sampling methods in a bidirectional manner.Specifically,we utilize the rays spawned from the sensors and also employ the rays emitted by the emitters to initialize the guiding structure.Results To demonstrate the benefits of our technique,we perform a comparative analysis of differentiable rendering and inverse rendering performance.We utilize a range of synthetic scene examples and evaluate our method against state-of-the-art projective-sampling-based differentiable rendering methods.Conclusions The experiments show that our method achieves lower variance gradients in the forward differentiable rendering process and better geometry reconstruction quality in the inverse-rendering results.
基金the National Natural Science Foundation of China(22003035,21963006,22073061)the Project of Shaanxi Province Youth Science and Technology New Star(2023KJXX-076)the National Training Program of Innovation and Entrepreneurship for Undergraduates(202314390018)。
文摘The utilization of phosphors that achieve full-spectrum lighting has emerged as a prevailing trend in the advancement of white light-emitting diode(WLED)lighting.In this study,we successfully prepared a novel green phosphor Ba_(2)Sc_(2)((BO_(3))_(2)B_(2)O_(5)):Ce^(3+)(BSBO:Ce^(3+))that can be utilized for full-spectrum lighting and low-temperature sensors.BSBO:Ce^(3+)exhibits a broad-band excitation spectrum centered at 410 nm,and a broad-band emission spectrum centered at 525 nm.The internal and external quantum efficiencies of BSBO:Ce^(3+)are 99%and 49%,respectively.The thermal stability of BSBO:Ce^(3+)can be improved by substituting partial Sc atoms with smaller cations.The thermal quenching mechanism of BSBO:Ce^(3+)and the lattice occupancy of Ce ions in BSBO are discussed in detail.Furthermore,by combining the green phosphor BSBO:Ce^(3+),the commercial blue phosphor and the red phosphor on a 405 nm chip,a white light source was obtained with a high average color rendering index(CRI)of 96.6,a low correlated color temperature(CCT)of 3988 K,and a high luminous efficacy of 88.0 Im/W.The lu-minous efficacy of the WLED exhibits negligible degradation during the 1000 h light aging experiment.What's more,an emission peak at 468 nm appears when excited at 352 nm and 80 K,however,the relative intensity of the peaks at 468 and 525 nm gradually weakens with increasing temperature,indicating the potential of this material as a low-temperature sensor.
文摘Ray casting algorithm can obtain a better quality image in volume rendering, however, it exists some problems, such as powerful computing capacity and slow rendering speed. How to improve the re-sampled speed is a key to speed up the ray casting algorithm. An algorithm is introduced to reduce matrix computation by matrix transformation characteristics of re-sampling points in a two coordinate system. The projection of 3-D datasets on image plane is adopted to reduce the number of rays. Utilizing boundary box technique avoids the sampling in empty voxel. By extending the Bresenham algorithm to three dimensions, each re-sampling point is calculated. Experimental results show that a two to three-fold improvement in rendering speed using the optimized algorithm, and the similar image quality to traditional algorithm can be achieved. The optimized algorithm can produce the required quality images, thus reducing the total operations and speeding up the volume rendering.
文摘As a cornerstone for applications such as autonomous driving,3D urban perception is a burgeoning field of study.Enhancing the performance and robustness of these perception systems is crucial for ensuring the safety of next-generation autonomous vehicles.In this work,we introduce a novel neural scene representation called Street Detection Gaussians(SDGs),which redefines urban 3D perception through an integrated architecture unifying reconstruction and detection.At its core lies the dynamic Gaussian representation,where time-conditioned parameterization enables simultaneous modeling of static environments and dynamic objects through physically constrained Gaussian evolution.The framework’s radar-enhanced perception module learns cross-modal correlations between sparse radardata anddense visual features,resulting ina22%reduction inocclusionerrors compared tovisiononly systems.A breakthrough differentiable rendering pipeline back-propagates semantic detection losses throughout the entire 3D reconstruction process,enabling the optimization of both geometric and semantic fidelity.Evaluated on the Waymo Open Dataset and the KITTI Dataset,the system achieves real-time performance(135 Frames Per Second(FPS)),photorealistic quality(Peak Signal-to-Noise Ratio(PSNR)34.9 dB),and state-of-the-art detection accuracy(78.1%Mean Average Precision(mAP)),demonstrating a 3.8×end-to-end improvement over existing hybrid approaches while enabling seamless integration with autonomous driving stacks.
基金supported by the Open Research Fund of Key Laboratory of Space Utilization,Chinese Academy of Sciences(No.LSU-YKZX-2017-02)
文摘This paper presents a new solution to haptic based teleoperation to control a large-sized slave robot for space exploration, which includes two specially designed haptic joysticks, a hybrid master-slave motion mapping method, and a haptic feedback model rendering the operating resistance and the interactive feedback on the slave side. Two devices using the 3 R and DELTA mechanisms respectively are developed to be manipulated to control the position and orientation of a large-sized slave robot by using both of a user's two hands respectively. The hybrid motion mapping method combines rate control and variable scaled position mapping to realize accurate and efficient master-slave control. Haptic feedback for these two mapping modes is designed with emphasis on ergonomics to improve the immersion of haptic based teleoperation. A stiffness estimation method is used to calculate the contact stiffness on the slave side and play the contact force rendered by using a traditional spring-damping model to a user on the master side stably. Experiments by using virtual environments to simulate the slave side are conducted to validate the effectiveness and efficiency of the proposed solution.
基金Project supported by the National Natural Science Foundation of China (No 60802013)the Natural Science Foundation of Zhe-jiang Province, China (No Y106574)
文摘A new algorithm is proposed for restoring disocclusion regions in depth-image-based rendering （DIBR） warped images. Current solutions include layered depth image （LDI）, pre-filtering methods, and post-processing methods. The LDI is complicated, and pre-filtering of depth images causes noticeable geometrical distortions in cases of large baseline warping. This paper presents a depth-aided inpainting method which inherits merits from Criminisi＇s inpainting algorithm. The proposed method features incorporation of a depth cue into texture estimation. The algorithm efficiently handles depth ambiguity by penalizing larger Lagrange multipliers of flling points closer to the warping position compared with the surrounding existing points. We perform morphological operations on depth images to accelerate the algorithm convergence, and adopt a luma-first strategy to adapt to various color sampling formats. Experiments on test multi-view sequence showed that our method has superiority in depth differentiation and geometrical loyalty in the restoration of warped images. Also, peak signal-to-noise ratio （PSNR） statistics on non-hole regions and whole image comparisons both compare favorably to those obtained by state of the art techniques.
