Resolution-enhanced multi-core fiber imaging learned on a digital twin for cancer diagnosis

Neurophotonics, Journal Year: 2024, Volume and Issue: 11(S1)

Published: Jan. 31, 2024

SignificanceDeep learning enables label-free all-optical biopsies and automated tissue classification. Endoscopic systems provide intraoperative diagnostics to deep speed up treatment without harmful removal. However, conventional multi-core fiber (MCF) endoscopes suffer from low resolution artifacts, which hinder tumor diagnostics.AimWe introduce a method enable unpixelated, high-resolution imaging through given MCF with diameter of around 0.65 mm arbitrary core arrangement inhomogeneous transmissivity.ApproachImage reconstruction is based on the digital twin concept single-reference-based simulation optical properties transfer small experimental dataset biological tissue. The reference provided physical information about during training processes.ResultsFor simulated data, hallucination caused by inhomogeneity was eliminated, averaged peak signal-to-noise ratio structural similarity were increased 11.2 dB 0.20 23.4 0.74, respectively. By learning, metrics independent test images experimentally acquired glioblastoma ex vivo can reach 31.6 0.97 14 fps computing speed.ConclusionsWith proposed approach, single image required in pre-training stage laborious acquisition data bypassed. Validation cryosections only 50 pairs showed capability for retrieval high clinical feasibility.

Language: Английский

---

Endoir: A GAN-based method for fiber bundle endoscope image restoration

Optics and Lasers in Engineering, Journal Year: 2024, Volume and Issue: 184, P. 108588 - 108588

Published: Sept. 18, 2024

Language: Английский

---

Honeycomb effect elimination in differential phase fiber-bundle-based endoscopy

Optics Express, Journal Year: 2024, Volume and Issue: 32(12), P. 20682 - 20682

Published: May 13, 2024

Fiber-bundle-based endoscopy, with its ultrathin probe and micrometer-level resolution, has become a widely adopted imaging modality for in vivo imaging. However, the fiber bundles introduce significant honeycomb effect, primarily due to multi-core structure crosstalk of adjacent cores, which superposes pattern image on original image. To tackle this issue, we propose an iterative-free spatial pixel shifting (SPS) algorithm, designed suppress effect enhance real-time performance. The process involves creation three additional sub-images by one at 0, 45, 90 degree angles. These four are then used compute differential maps x y directions. By performing spiral integration these maps, reconstruct honeycomb-free improved details. Our simulations experimental results, conducted self-built bundle-based endoscopy system, demonstrate effectiveness SPS algorithm. significantly improves quality reflective objects unlabeled transparent scattered objects, laying solid foundation biomedical endoscopic applications.

Language: Английский

---

Optical Neural Network in Free-Space and Nanophotonics

IEEE Access, Journal Year: 2023, Volume and Issue: 11, P. 88656 - 88669

Published: Jan. 1, 2023

The explosive data growth has resulted in increased computing costs. As Moore's Law is increasingly slowing down, the traditional approach based on von Neumann architecture gradually becoming unable to fulfill future needs. However, optical neural networks have emerged as a potential solution because of their high speed, bandwidth, and capability subdue bottleneck problem power. With development optics nanophotonics, it possible implement complex free-space nanophotonic platforms. This article reviews research progress networks. Firstly, various methods implementing matrix calculations are described. Secondly, construction method network free space platform introduced respectively. In space, 4f system diffractive elements nanophotonic, relied waveguide devices such microring resonator or Mach-Zehnder Interferometer. Thirdly, we introduce training nonlinear activity. Finally, summarized current status challenges future, great application value.

Language: Английский

---

Step-by-step transfer function reversal for single-shot 3D fiber endoscopy using a diffuser

Published: March 15, 2023

Endoscopy through coherent fiber bundles plays a significant role in industrial and medical 2D imaging. By replacing the lens on distal side with diffuser, 3D information of measurement volume is encoded as speckle patterns camera. Neural networks can then be employed to reconstruct object. Therefore, minimally invasive single-shot imaging flexible low-cost endoscope diameter less than 1 mm enabled. However, number cores limiting transferable reduces reconstruction quality. In this paper, separate for diffuser bundle different core numbers explored. This approach enables biomedical applications vivo diagnostics, e.g. fluorescence 3D.

Language: Английский

---

Real-Time Endomicroscopic Image Mosaicking with an EKF-based Sensor Fusion Approach

2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Journal Year: 2023, Volume and Issue: unknown, P. 1 - 5

Published: July 24, 2023

This study presents a real-time sensor fusion framework based on the extended Kalman filter (EKF) for accurate and robust endomicroscopic image mosaicking. The incorporates an optical tracking system that can track 6-DOF pose of imaging probe with high accuracy in real time conjunction 2D local registration from feature matching between two consecutive frames. We evaluated performance mosaicking compared or tracker only approach. As result, it could retain microscopic level detail image-based approach also achieve mosaic without any drift by using system.

Language: Английский

---