Learning to Deblur using Light Field Generated and Real Defocus Images

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Journal Year: 2022, Volume and Issue: unknown, P. 16283 - 16292

Published: June 1, 2022

Defocus deblurring is a challenging task due to the spatially varying nature of defocus blur. While deep learning approach shows great promise in solving image restoration problems, demands accurate training data that consists all-in-focus and pairs, which difficult collect. Naive two-shot capturing cannot achieve pixel-wise correspondence between defocused pairs. Synthetic aperture light fields suggested be more reliable way generate However, blur generated from field different images captured with traditional digital camera. In this paper, we propose novel network leverages strength overcomes shortcoming fields. We first train on field-generated dataset for its highly correspondence. Then, fine-tune using feature loss another collected by method alleviate differences exists two domains. This strategy proved effective able state-of-the-art performance both quantitatively qualitatively multiple test sets. Extensive ablation studies have been conducted analyze effect each module final performance.

Language: Английский

---

Deep Learning for Camera Autofocus

IEEE Transactions on Computational Imaging, Journal Year: 2021, Volume and Issue: 7, P. 258 - 271

Published: Jan. 1, 2021

Most digital cameras use specialized autofocus sensors, such as phase detection, lidar or ultrasound, to directly measure focus state. However, sensors increase cost and complexity without optimizing final image quality. This paper proposes a new pipeline for image-based shows that neural analysis finds 5-10x faster than traditional contrast enhancement. We achieve this by learning the direct mapping between an its position. In further with conventional methods, AI methods can generate scene-based trajectories optimize synthesized quality dynamic three dimensional scenes. propose control strategy varies focal position dynamically maximize estimated from stack. rule-based agent learned different scenarios show their advantages over other stacking methods.

Language: Английский

---

Joint Depth and Defocus Estimation From a Single Image Using Physical Consistency

IEEE Transactions on Image Processing, Journal Year: 2021, Volume and Issue: 30, P. 3419 - 3433

Published: Jan. 1, 2021

Estimating depth and defocus maps are two fundamental tasks in computer vision. Recently, many methods explore these separately with the help of powerful feature learning ability deep have achieved impressive progress. However, due to difficulty densely labeling on real images, mostly based synthetic training dataset, performance learned network degrades significantly images. In this paper, we tackle a new task that jointly estimates from single image. We design dual subnets respectively for estimating defocus. The is trained dataset physical constraint enforce consistency between Moreover, simple method label order image novel metrics measure accuracies estimation Comprehensive experiments demonstrate joint using enables guide each other, effectively improves their defocused dataset.

Language: Английский

---

Bridging Unsupervised and Supervised Depth from Focus via All-in-Focus Supervision

2021 IEEE/CVF International Conference on Computer Vision (ICCV), Journal Year: 2021, Volume and Issue: unknown, P. 12601 - 12611

Published: Oct. 1, 2021

Depth estimation is a long-lasting yet important task in computer vision. Most of the previous works try to estimate depth from input images and assume are all-in-focus (AiF), which less common real-world applications. On other hand, few take defocus blur into account consider it as another cue for estimation. In this paper, we propose method not only map but an AiF image set with different focus positions (known focal stack). We design shared architecture exploit relationship between As result, proposed can be trained either supervisedly ground truth depth, or unsupervisedly supervisory signals. show various experiments that our outperforms state-of-the-art methods both quantitatively qualitatively, also has higher efficiency inference time.

Language: Английский

---

Deep Learning-Based Dynamic Region of Interest Autofocus Method for Grayscale Image

Sensors, Journal Year: 2024, Volume and Issue: 24(13), P. 4336 - 4336

Published: July 4, 2024

In the field of autofocus for optical systems, although passive focusing methods are widely used due to their cost-effectiveness, fixed windows and evaluation functions in certain scenarios can still lead failures. Additionally, lack datasets limits extensive research deep learning methods. this work, we propose a neural network method with capability dynamically selecting region interest (ROI). Our main work is as follows: first, construct dataset automatic grayscale images; second, transform issue into an ordinal regression problem two strategies: full-stack search single-frame prediction; third, MobileViT linear self-attention mechanism achieve on dynamic regions interest. The effectiveness proposed verified through experiments, results show that MAE be low 0.094, time 27.8 ms, prediction 0.142, 27.5 ms.

Language: Английский

---

DC²: Dual-Camera Defocus Control by Learning to Refocus

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Journal Year: 2023, Volume and Issue: unknown, P. 21488 - 21497

Published: June 1, 2023

Smartphone cameras today are increasingly approaching the versatility and quality of professional through a combination hardware software advancements. However, fixed aperture remains key limitation, preventing users from controlling depth field (DoF) captured images. At same time, many smartphones now have multiple with different apertures - specifically, an ultra-wide camera wider view deeper DoF higher resolution primary shallower DoF. In this work, we propose $DC^{2}$ , system for defocus control synthetically varying aperture, focus distance arbitrary effects by fusing information such dual-camera system. Our insight is to leverage real-world smartphone dataset using image refocus as proxy task learning defocus. Quantitative qualitative evaluations on data demonstrate our system's efficacy where outperform state-of-the-art deblurring, bokeh rendering, refocus. Finally, creative post-capture enabled method, including tilt-shift content-based effects.

Language: Английский

---

Microscopic Image Deblurring by a Generative Adversarial Network for 2D Nanomaterials: Implications for Wafer-Scale Semiconductor Characterization

ACS Applied Nano Materials, Journal Year: 2022, Volume and Issue: 5(9), P. 12855 - 12864

Published: Sept. 12, 2022

Wafer-scale two-dimensional (2D) semiconductors with atomically thin layers are promising materials for fabricating optic and photonic devices. Bright-field microscopy is a widely utilized method large-area characterization, layer number identification, quality assessment of 2D based on optical contrast. Out-of-focus microscopic images caused by instrumental focus drifts contained blurred degraded structural color information, hindering the reliability automated identification nanosheets. To achieve restoration accurate deep-learning-based imagery deblurring (MID) was developed. Specifically, generative adversarial network an improved loss function employed to recover both information out-of-focus low-quality images. MoS2 grown chemical vapor deposition SiO2/Si substrate characterized. Quantitative indexes including similarity (SSIM), peak signal-to-noise ratio, CIE 1931 space were studied evaluate performance MID images, minimum value SSIM over 90% deblurred Further, pre-trained U-Net model average accuracy 80% implemented segment predict distribution nanosheet categories (monolayer, bilayer, trilayer, multi-layer, bulk). The developed image using allow on-site, accurate, characterization analyzing local properties. This may be in wafer-scale industrial manufacturing monitoring

Language: Английский

---

Lens Parameter Estimation for Realistic Depth of Field Modeling

2021 IEEE/CVF International Conference on Computer Vision (ICCV), Journal Year: 2023, Volume and Issue: unknown

Published: Oct. 1, 2023

We present a method to estimate the depth of field effect from single image. Most existing methods related this task provide either per-pixel estimation blur and/or depth. Instead, we go further and propose use lens-based representation that models using two parameters: factor focus disparity. Those parameters, along with signed defocus representation, result in more intuitive linear which solve novel weighting network. Furthermore, our explicitly enforces consistency between estimated blur, lens map. Finally, train deep-learning-based model on mix real images synthetic fully images. These improvements robust accurate method, as demonstrated by state-of-the-art results. In particular, parametrization enables several applications, such 3D staging for AR environments seamless object compositing.

Language: Английский

---

Defocus Magnification Using Conditional Adversarial Networks

Published: Jan. 1, 2019

Defocus magnification is the process of rendering a shallow depth-of-field in an image captured using camera with narrow aperture. useful tool photography for emphasis on subject and highlighting background bokeh. Estimating per-pixel blur kernel or depth-map scene followed by spatially-varying re-blurring standard approach to defocus magnification. We propose single-step that directly converts narrow-aperture wide-aperture image. use conditional adversarial network trained multi-aperture images created from light-fields. novel loss term based composite focus measure improve generalization show high quality

Language: Английский

---

Time-Division Multiplexing Light Field Display With Learned Coded Aperture

IEEE Transactions on Image Processing, Journal Year: 2022, Volume and Issue: 32, P. 350 - 363

Published: Dec. 15, 2022

Conventional stereoscopic displays suffer from vergence-accommodation conflict and cause visual fatigue. Integral-imaging-based resolve the problem by directly projecting sub-aperture views of a light field into eyes using microlens array or similar structure. However, such have an inherent trade-off between angular spatial resolutions. In this paper, we propose novel coded time-division multiplexing technique that projects encoded to viewer with correct cues for reflex. Given sparse views, our pipeline can provide perception high-resolution refocused images minimal aliasing jointly optimizing display aperture pattern. This is achieved via deep learning in end-to-end fashion simulating transport image formation Fourier optics. To knowledge, work among first optimize learning. We verify idea objective quality metrics (PSNR, SSIM, LPIPS) perform extensive study on various customizable design variables pipeline. Experimental results show fields displayed proposed indeed higher than baseline designs.

Language: Английский

---