NHVC: Neural Holographic Video Compression with Scalable Architecture DOI

Hyunmin Ban,

Seung-Mi Choi,

Jun Yeong

и другие.

Опубликована: Март 16, 2024

Recently, neural network-based approaches for hologram generation and compression have gained popularity as they allow efficient inference on GPUs without the need iterative optimization required in traditional methods. In this paper, we introduce Neural Holographic Video Compression (NHVC), an end-to-end trainable scalable model designed high-quality phase video compression. NHVC consists of auto-encoder-based generator, a latent coder and-two hyper-prior coders. For each input image, features are extracted through encoder part generator then entropy coded at shared based information. The two coders employ spatial spatio-temporal I-frames P-frames, respectively. With architecture, our can offer task-scalability, allowing single trained to serve image compressor, or compressor required.Experimental results with UVG dataset show that outperforms 'HoloNet + VVC' by 75.6% BD-Rate reduction, modest 2K encoding decoding speeds (5 fps 12 fps, respectively). task, showed much higher-quality (almost 42dB PSNR) reconstruction using dataset, while previous HoloNet provides most 36dB quality. We also provide extensive experimental study several important design questions such quadruple extension (QE) model, feasibility motion estimation domain, alternative, increasing receptive field learn better features, variable rate support model. It is noteworthy first best providing task-scalability.

Язык: Английский

Full-colour 3D holographic augmented-reality displays with metasurface waveguides DOI Creative Commons
Manu Gopakumar, Gun‐Yeal Lee, Suyeon Choi

и другие.

Nature, Год журнала: 2024, Номер 629(8013), С. 791 - 797

Опубликована: Май 8, 2024

Emerging spatial computing systems seamlessly superimpose digital information on the physical environment observed by a user, enabling transformative experiences across various domains, such as entertainment, education, communication and training

Язык: Английский

Процитировано

89

A review of liquid crystal spatial light modulators: devices and applications DOI Creative Commons
Yiqian Yang, Andrew Forbes, Liangcai Cao

и другие.

Opto-Electronic Science, Год журнала: 2023, Номер 2(8), С. 230026 - 230026

Опубликована: Янв. 1, 2023

Spatial light modulators, as dynamic flat-panel optical devices, have witnessed rapid development over the past two decades, concomitant with advancements in micro- and opto-electronic integration technology. In particular, liquid-crystal spatial modulator (LC-SLM) technologies been regarded versatile tools for generating arbitrary fields tailoring all degrees of freedom beyond just phase amplitude. These devices gained significant interest nascent field structured space time, facilitated by their ease use real-time manipulation, fueling both fundamental research practical applications. Here we provide an overview key working principles LC-SLMs review progress made to date deployment various applications, covering topics diverse beam shaping steering, holography, trapping tweezers, measurement, wavefront coding, vortex, quantum optics. Finally, conclude outlook on potential opportunities technical challenges this rapidly developing field.

Язык: Английский

Процитировано

56

Waveguide holography for 3D augmented reality glasses DOI Creative Commons
Changwon Jang, Kiseung Bang, Minseok Chae

и другие.

Nature Communications, Год журнала: 2024, Номер 15(1)

Опубликована: Янв. 2, 2024

Abstract Near-eye displays are fundamental technology in the next generation computing platforms for augmented reality and virtual reality. However, there remaining challenges to deliver immersive comfortable visual experiences users, such as compact form factor, solving vergence-accommodation conflict, achieving a high resolution with large eyebox. Here we show holographic near-eye display concept that combines advantages of waveguide overcome towards true 3D glasses. By modeling coherent light interactions propagation via combiner, demonstrate controlling output wavefront using spatial modulator located at input coupler side. The proposed method enables exit-pupil expanding combiners, providing software-steerable It also offers additional enhancement capability by suppressing phase discontinuities caused pupil replication process. We build prototypes verify experimental results conclude paper discussion.

Язык: Английский

Процитировано

40

Non-convex optimization for inverse problem solving in computer-generated holography DOI Creative Commons
Xiaomeng Sui, Zehao He, Daping Chu

и другие.

Light Science & Applications, Год журнала: 2024, Номер 13(1)

Опубликована: Июль 9, 2024

Abstract Computer-generated holography is a promising technique that modulates user-defined wavefronts with digital holograms. Computing appropriate holograms faithful reconstructions not only problem closely related to the fundamental basis of but also long-standing challenge for researchers in general fields optics. Finding exact solution desired hologram reconstruct an accurate target object constitutes ill-posed inverse problem. The practice single-diffraction computation synthesizing can provide approximate answer, which subject limitations numerical implementation. Various non-convex optimization algorithms are thus designed seek optimal by introducing different constraints, frameworks, and initializations. Herein, we overview applied computer-generated holography, incorporating principles synthesis based on alternative projections gradient descent methods. This aimed underlying optimized generation, as well insights into cutting-edge developments this rapidly evolving field potential applications virtual reality, augmented head-up display, data encryption, laser fabrication, metasurface design.

Язык: Английский

Процитировано

17

Ultrahigh-fidelity full-color holographic display via color-aware optimization DOI Creative Commons
Chun Chen, Seung‐Woo Nam, Dongyeon Kim

и другие.

PhotoniX, Год журнала: 2024, Номер 5(1)

Опубликована: Июнь 1, 2024

Abstract Holographic display offers the capability to generate high-quality images with a wide color gamut since it is laser-driven. However, many existing holographic techniques fail fully exploit this potential, primarily due system’s imperfections. Such flaws often result in inaccurate representation, and there lack of an efficient way address accuracy issue. In study, we develop color-aware hologram optimization approach for color-accurate displays. Our integrates both laser camera into loop, enabling dynamic laser’s output acquisition physically captured feedback. Moreover, improve efficiency process video We introduce cascade strategy, which leverages redundant neighbor information accelerate iterative process. evaluate our method through simulation optical experiments, demonstrating superiority terms image quality, accuracy, speed compared previous algorithms. verifies promising realize high-fidelity display, provides new direction toward practical display.

Язык: Английский

Процитировано

10

HoloChrome: Polychromatic Illumination for Speckle Reduction in Holographic Near-Eye Displays DOI
Florian Schiffers, Grace Kuo, Nathan Matsuda

и другие.

ACM Transactions on Graphics, Год журнала: 2025, Номер unknown

Опубликована: Апрель 28, 2025

Holographic displays hold the promise of providing authentic depth cues, resulting in enhanced immersive visual experiences for near-eye applications. However, current holographic are hindered by speckle noise, which limits accurate reproduction color and texture displayed images. We present HoloChrome, a polychromatic display framework designed to mitigate these limitations. HoloChrome utilizes an ultrafast, wavelength-adjustable laser dual-Spatial Light Modulator (SLM) architecture, enabling multiplexing large set discrete wavelengths across visible spectrum. By leveraging spatial separation our dual-SLM setup, we independently manipulate patterns multiple wavelengths. This novel approach effectively reduces noise through incoherent averaging achieved wavelength multiplexing, specifically using single SLM pattern modulate simultaneously on one or more devices. Our method is complementary existing reduction techniques, offering new pathway address this challenge. Furthermore, use illumination broadens achievable gamut compared traditional three-color primary displays. simulations tabletop experiments validate that significantly expands gamut. These advancements enhance performance displays, moving us closer practical, next-generation experiences.

Язык: Английский

Процитировано

1

Holographic Glasses for Virtual Reality DOI
Jonghyun Kim, Manu Gopakumar, Suyeon Choi

и другие.

Опубликована: Июль 20, 2022

We present Holographic Glasses, a holographic near-eye display system with an eyeglasses-like form factor for virtual reality. Glasses are composed of pupil-replicating waveguide, spatial light modulator, and geometric phase lens to create images in lightweight thin factor. The proposed design can deliver full-color 3D using optical stack 2.5 mm thickness. A novel pupil-high-order gradient descent algorithm is presented the correct calculation user's varying pupil size. implement benchtop wearable prototypes testing. Our binocular prototype supports focus cues provides diagonal field view 22.8° 2.3 static eye box additional capabilities dynamic beam steering, while weighing only 60 g excluding driving board.

Язык: Английский

Процитировано

36

Diffraction model-informed neural network for unsupervised layer-based computer-generated holography DOI Creative Commons

Xinghua Shui,

Huadong Zheng, Xinxing Xia

и другие.

Optics Express, Год журнала: 2022, Номер 30(25), С. 44814 - 44814

Опубликована: Ноя. 10, 2022

Learning-based computer-generated holography (CGH) has shown remarkable promise to enable real-time holographic displays. Supervised CGH requires creating a large-scale dataset with target images and corresponding holograms. We propose diffraction model-informed neural network framework (self-holo) for 3D phase-only hologram generation. Due the angular spectrum propagation being incorporated into network, self-holo can be trained in an unsupervised manner without need of labeled dataset. Utilizing various representations object randomly reconstructing one layer keeps complexity independent number depth layers. The takes amplitude map as input synthesizes or 2D hologram. demonstrate reconstructions good effect generalizability numerical optical experiments.

Язык: Английский

Процитировано

29

Real-time High-Quality Computer-Generated Hologram Using Complex-Valued Convolutional Neural Network DOI
Chongli Zhong, Xinzhu Sang,

Binbin Yan

и другие.

IEEE Transactions on Visualization and Computer Graphics, Год журнала: 2023, Номер 30(7), С. 3709 - 3718

Опубликована: Янв. 25, 2023

Holographic displays are ideal display technologies for virtual and augmented reality because all visual cues provided. However, real-time high-quality holographic difficult to achieve the generation of computer-generated hologram (CGH) is inefficient in existing algorithms. Here, complex-valued convolutional neural network (CCNN) proposed phase-only CGH generation. The CCNN-CGH architecture effective with a simple structure based on character design complex amplitude. A prototype set up optical reconstruction. Experiments verify that state-of-the-art performance achieved terms quality speed end-to-end holography methods using wave propagation model. three times faster than HoloNet one-sixth Holo-encoder, Peak Signal Noise Ratio (PSNR) increased by 3 dB 9 dB, respectively. Real-time CGHs generated 1920 × 1072 3840 2160 resolutions dynamic displays.

Язык: Английский

Процитировано

21

Holographic Parallax Improves 3D Perceptual Realism DOI
Dongyeon Kim, Seung‐Woo Nam, Suyeon Choi

и другие.

ACM Transactions on Graphics, Год журнала: 2024, Номер 43(4), С. 1 - 13

Опубликована: Июль 19, 2024

Holographic near-eye displays are a promising technology to solve long-standing challenges in virtual and augmented reality display systems. Over the last few years, many different computer-generated holography (CGH) algorithms have been proposed that supervised by types of target content, such as 2.5D RGB-depth maps, 3D focal stacks, 4D light fields. It is unclear, however, what perceptual implications choice algorithm content type. In this work, we build testbed full-color, high-quality holographic display. Under natural viewing conditions, examine effects various CGH supervision formats conduct user studies assess their impacts on realism. Our results indicate designed for specific viewpoints exhibit noticeable deficiencies achieving contrast, holograms incorporating parallax cues consistently outperform other across including center eyebox. This finding particularly interesting suggests inclusion rendering plays crucial role enhancing overall quality experience. work represents an initial stride towards delivering perceptually realistic experience with displays.

Язык: Английский

Процитировано

8