EventLFM: event camera integrated Fourier light field microscopy for ultrafast 3D imaging

Light Science & Applications, Год журнала: 2024, Номер 13(1)

Опубликована: Июнь 26, 2024

Abstract Ultrafast 3D imaging is indispensable for visualizing complex and dynamic biological processes. Conventional scanning-based techniques necessitate an inherent trade-off between acquisition speed space-bandwidth product (SBP). Emerging single-shot wide-field offer a promising alternative but are bottlenecked by the synchronous readout constraints of conventional CMOS systems, thus restricting data throughput to maintain high SBP at limited frame rates. To address this, we introduce EventLFM, straightforward cost-effective system that overcomes these challenges integrating event camera with Fourier light field microscopy (LFM), state-of-the-art technique. The operates on novel asynchronous architecture, thereby bypassing rate limitations systems. We further develop simple robust event-driven LFM reconstruction algorithm can reliably reconstruct dynamics from unique spatiotemporal measurements captured EventLFM. Experimental results demonstrate EventLFM robustly fast-moving rapidly blinking fluorescent samples kHz Furthermore, highlight EventLFM’s capability neuronal signals in scattering mouse brain tissues tracking GFP-labeled neurons freely moving C. elegans . believe combined ultrafast large offered may open up new possibilities across many biomedical applications.

Язык: Английский

---

Deep-learning-augmented computational miniature mesoscope

Optica, Год журнала: 2022, Номер 9(9), С. 1009 - 1009

Опубликована: Авг. 3, 2022

Fluorescence microscopy is essential to study biological structures and dynamics. However, existing systems suffer from a tradeoff between field-of-view (FOV), resolution, complexity, thus cannot fulfill the emerging need of miniaturized platforms providing micron-scale resolution across centimeter-scale FOVs. To overcome this challenge, we developed Computational Miniature Mesoscope (CM$^2$) that exploits computational imaging strategy enable single-shot 3D high-resolution wide FOV in platform. Here, present CM$^2$ V2 significantly advances both hardware computation. We complement 3$\times$3 microlens array with new hybrid emission filter improves contrast by 5$\times$, design 3D-printed freeform collimator for LED illuminator excitation efficiency 3$\times$. reconstruction large volume, develop an accurate efficient linear shift-variant (LSV) model characterizes spatially varying aberrations. then train multi-module deep learning model, CM$^2$Net, using only 3D-LSV simulator. show CM$^2$Net generalizes well experiments achieves $\sim$7-mm 800-$\mu$m depth, provides $\sim$6-$\mu$m lateral $\sim$25-$\mu$m axial resolution. This $\sim$8$\times$ better localization $\sim$1400$\times$ faster speed as compared previous model-based algorithm. anticipate simple low-cost miniature system will be impactful many large-scale fluorescence applications.

Язык: Английский

---

Mesoscopic calcium imaging in a head-unrestrained male non-human primate using a lensless microscope

Nature Communications, Год журнала: 2024, Номер 15(1)

Опубликована: Фев. 10, 2024

Mesoscopic calcium imaging enables studies of cell-type specific neural activity over large areas. A growing body literature suggests that can be different when animals are free to move compared they restrained. Unfortunately, existing systems for dynamics areas in non-human primates (NHPs) table-top devices require restraint the animal's head. Here, we demonstrate an device capable mesoscale a head-unrestrained male primate. We successfully miniaturize our system by replacing lenses with optical mask and computational algorithms. The resulting lensless microscope fit comfortably on NHP, allowing its head freely while imaging. able measure orientation columns maps 20 mm2 field-of-view macaque. Our work establishes mesoscopic using as powerful approach studying under more naturalistic conditions.

Язык: Английский

---

Increasing the spatial bandwidth product in Light Field Microscopy with remote scanning

Biomedical Optics Express, Год журнала: 2025, Номер unknown

Опубликована: Янв. 24, 2025

Achieving fast, large-scale volumetric imaging with micrometer resolution has been a persistent challenge in biological microscopy. To address this challenge, we report an augmented version of light field microscopy, incorporating motorized tilting mirror upstream the camera. Depending on scanning pattern, view and/or lateral can be greatly improved. Our microscope technique is simple, versatile, and configured for bright-field epifluorescence modes. We demonstrate its performance multi-cellular aggregates various shapes sizes.

Язык: Английский

---

Large-FOV 3D localization microscopy by spatially variant point spread function generation

Science Advances, Год журнала: 2024, Номер 10(10)

Опубликована: Март 8, 2024

Accurate characterization of the microscopic point spread function (PSF) is crucial for achieving high-performance localization microscopy (LM). Traditionally, LM assumes a spatially invariant PSF to simplify modeling imaging system. However, large fields view (FOV) imaging, it becomes important account variant nature PSF. Here, we propose an accurate and fast principal components analysis–based field-dependent 3D generator (PPG3D) localizer LM. Through simulations experimental three-dimensional (3D) single-molecule (SMLM), demonstrate effectiveness PPG3D, enabling super-resolution mitochondria microtubules with high fidelity over FOV. A comparison PPG3D shift-variant reveals threefold improvement in accuracy. Moreover, approximately 100 times faster than existing generators, when used image plane–based interpolation mode. Given its user-friendliness, believe that holds great potential widespread application SMLM other modalities.

Язык: Английский

---

Multiple-scattering simulator-trained neural network for intensity diffraction tomography

Optics Express, Год журнала: 2023, Номер 31(3), С. 4094 - 4094

Опубликована: Янв. 6, 2023

Recovering 3D phase features of complex biological samples traditionally sacrifices computational efficiency and processing time for physical model accuracy reconstruction quality. Here, we overcome this challenge using an approximant-guided deep learning framework in a high-speed intensity diffraction tomography system. Applying physics simulator-based strategy trained entirely on natural image datasets, show our network can robustly reconstruct samples. To achieve highly efficient training prediction, implement lightweight 2D structure that utilizes multi-channel input encoding the axial information. We demonstrate experimental measurements weakly scattering epithelial buccal cells strongly C. elegans worms. benchmark network’s performance against state-of-the-art multiple-scattering model-based iterative algorithm. highlight robustness by reconstructing dynamic from living worm video. further emphasize generalization capabilities recovering algae imaged different setups. assess prediction quality, develop quantitative evaluation metric to predictions are consistent with both measurements.

Язык: Английский

---

Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformers

IEEE Transactions on Image Processing, Год журнала: 2024, Номер 33, С. 4568 - 4583

Опубликована: Янв. 1, 2024

High-quality panoramic images with a Field of View (FoV) 360° are essential for contemporary computer vision tasks. However, conventional imaging systems come sophisticated lens designs and heavy optical components. This disqualifies their usage in many mobile wearable applications where thin portable, minimalist desired. In this paper, we propose Panoramic Computational Imaging Engine (PCIE) to achieve high-quality imaging. With less than three spherical lenses, Minimalist Prototype (MPIP) is constructed based on the design Annular Lens (PAL), but low-quality results due aberrations small image plane size. We two pipelines, i.e. Aberration Correction (AC) Super-Resolution (SR&AC), solve quality problems MPIP, sensors large pixel size, respectively. To leverage prior information system, Point Spread Function (PSF) representation method produce PSF map as an additional modality. A PSF-aware Aberration-image Recovery Transformer (PART) designed universal network which self-attention calculation feature extraction guided by map. train PART synthetic pairs from simulation put forward PALHQ dataset fill gap real-world PAL low-level vision. comprehensive variety experiments benchmarks demonstrates impressive PCIE effectiveness representation. further deliver heuristic experimental findings imaging, terms choices prototype pipeline, architecture, training strategies, construction. Our code will be available at https://github.com/zju-jiangqi/PCIE-PART.

Язык: Английский

---

A miniaturized mesoscope for the large-scale single-neuron-resolved imaging of neuronal activity in freely behaving mice

Nature Biomedical Engineering, Год журнала: 2024, Номер 8(6), С. 754 - 774

Опубликована: Июнь 20, 2024

Язык: Английский

---

Real-time, deep-learning aided lensless microscope

Biomedical Optics Express, Год журнала: 2023, Номер 14(8), С. 4037 - 4037

Опубликована: Июль 4, 2023

Traditional miniaturized fluorescence microscopes are critical tools for modern biology. Invariably, they struggle to simultaneously image with a high spatial resolution and large field of view (FOV). Lensless offer solution this limitation. However, real-time visualization samples is not possible lensless imaging, as reconstruction can take minutes complete. This poses challenge usability, crucial feature that assists users in identifying locating the imaging target. The issue particularly pronounced operate at close distances. Imaging distances requires shift-varying deconvolution account variation point spread function (PSF) across FOV. Here, we present microscope achieves by eliminating use an iterative algorithm. neural network-based method show here, more than 10000 times increase speed compared reconstruction. increased allows us visualize results our 25 frames per second (fps), while achieving better 7 µm over FOV 10 mm

Язык: Английский

---

Wide-field, high-resolution reconstruction in computational multi-aperture miniscope using a Fourier neural network

Optica, Год журнала: 2024, Номер 11(6), С. 860 - 860

Опубликована: Май 28, 2024

Traditional fluorescence microscopy is constrained by inherent trade-offs among resolution, field of view, and system complexity. To navigate these challenges, we introduce a simple low-cost computational multi-aperture miniature microscope, utilizing microlens array for single-shot wide-field, high-resolution imaging. Addressing the challenges posed extensive view multiplexing non-local, shift-variant aberrations in this device, present SV-FourierNet, multi-channel Fourier neural network. SV-FourierNet facilitates image reconstruction across entire imaging through its learned global receptive field. We establish close relationship between physical spatially varying point-spread functions network's effective This ensures that has effectively encapsulated our physically meaningful function reconstruction. Training conducted entirely on physics-based simulator. showcase video reconstructions colonies freely moving C. elegans mouse brain section. Our augmented with represents major advancement may find broad applications biomedical research other fields requiring compact solutions.

Язык: Английский

---