Data leakage in deep learning studies of translational EEG

Frontiers in Neuroscience, Journal Year: 2024, Volume and Issue: 18

Published: May 3, 2024

A growing number of studies apply deep neural networks (DNNs) to recordings human electroencephalography (EEG) identify a range disorders. In many studies, EEG are split into segments, and each segment is randomly assigned the training or test set. As consequence, data from individual subjects appears in both Could high test-set accuracy reflect leakage subject-specific patterns data, rather than that disease? We address this question by testing performance DNN classifiers using segment-based holdout (in which segments one subject can appear set), comparing their subject-based (where all exclusively either set set). two datasets (one classifying Alzheimer's disease, other epileptic seizures), we find on previously-unseen strongly overestimated when models trained holdout. Finally, survey literature majority translational DNN-EEG use Most published may dramatically overestimate classification new subjects.

Language: Английский

---

Analyzing Information Leakage on Video Object Detection Datasets by Splitting Images Into Clusters With High Spatiotemporal Correlation

IEEE Access, Journal Year: 2024, Volume and Issue: 12, P. 47646 - 47655

Published: Jan. 1, 2024

Random splitting strategy is a common approach for training, testing, and validating object detection algorithms based on deep learning. Is datasets to have images extracted from video sources, in which there are frames with high spatial correlation, i.e., rotated positions or different view angles of the same object. These highly correlated may lead information leakage if these not well-distributed. In this work, it shown that created correlation using random distribute image into sub-datasets. It proposed clustering dataset split algorithm distributed randomly sub-datasets pack clusters instead single at time. The by extracting features an image-text pre-trained model, CLIP, reducing feature vector dimensionality t-Distributed Stochastic Neighbor embedding (t-SNE). reduced dimensional representation, separated like DBSCAN, OPTICS, Agglomerative Clustering. train, test, validation avoiding frames. YOLOv8 used as detector test splitting.

Language: Английский

---

Multi-site cross-organ calibrated deep learning (MuSClD): Automated diagnosis of non-melanoma skin cancer

Medical Image Analysis, Journal Year: 2022, Volume and Issue: 84, P. 102702 - 102702

Published: Nov. 24, 2022

Although deep learning (DL) has demonstrated impressive diagnostic performance for a variety of computational pathology tasks, this often markedly deteriorates on whole slide images (WSI) generated at external test sites. This phenomenon is due in part to domain shift, wherein differences test-site pre-analytical variables (e.g., scanner, staining procedure) result WSI with notably different visual presentations compared training data. To ameliorate pre-analytic variances, approaches such as CycleGAN can be used calibrate properties between sites, the intent improving DL classifier generalizability. In work, we present new approach termed Multi-Site Cross-Organ Calibration based Deep Learning (MuSClD) that employs WSIs an off-target organ calibration created same site on-target organ, off assumption cross-organ slides are subjected common set sources variance. We demonstrate by using from data, shift and testing data mitigated. Importantly, strategy uniquely guards against potential leakage introduced during calibration, information only available imparted evaluate MuSClD context automated diagnosis non-melanoma skin cancer (NMSC). Specifically, evaluated identifying distinguishing (a) basal cell carcinoma (BCC), (b) in-situ squamous carcinomas (SCC-In Situ), (c) invasive (SCC-Invasive), Australian (training, n = 85) Swiss (held-out testing, 352) cohort. Our experiments reveal MuSCID reduces Wasserstein distances sites terms color, contrast, brightness metrics, without imparting noticeable artifacts The NMSC-subtyping statistically improved one-vs. rest AUC: BCC (0.92 vs 0.87, p 0.01), SCC-In Situ (0.87 0.73, 0.15) SCC-Invasive 0.82, 1e-5). Compared baseline no internal validation results (BCC (0.98), (0.92), (0.97)) suggest while indeed degrades classification performance, our tissue safely compensate variabilities, robustness model.

Language: Английский

---

Vision Transformers and Transfer Learning Approaches for Arabic Sign Language Recognition

Applied Sciences, Journal Year: 2023, Volume and Issue: 13(21), P. 11625 - 11625

Published: Oct. 24, 2023

Sign languages are complex, but there ongoing research efforts in engineering and data science to recognize, understand, utilize them real-time applications. Arabic sign language recognition (ArSL) has been examined applied using various traditional intelligent methods. However, have limited attempts enhance this process by utilizing pretrained models large-sized vision transformers designed for image classification tasks. This study aimed create robust transfer learning trained on a dataset of 54,049 images depicting 32 alphabets from an ArSL dataset. The goal was accurately classify these into their corresponding alphabets. included two methodological parts. first one the approach, wherein we utilized namely MobileNet, Xception, Inception, InceptionResNet, DenseNet, BiT, ViT, Swin. We evaluated different variants base-sized with weights initialized ImageNet or otherwise randomly. second part deep approach convolutional neural networks (CNNs), several CNN architectures were scratch be compared approach. proposed methods accuracy, AUC, precision, recall, F1 loss metrics. consistently performed well outperformed other models. ResNet InceptionResNet obtained comparably high performance 98%. By combining concepts transformer-based architecture pretraining, ViT Swin leveraged strengths both reduced number parameters required training, making more efficient stable than existing studies classification. demonstrates effectiveness robustness low-resourced languages.

Language: Английский

---

Localization is All You Evaluate: Data Leakage in Online Mapping Datasets and How to Fix it

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Journal Year: 2024, Volume and Issue: 9, P. 22150 - 22159

Published: June 16, 2024

Language: Английский

---

Unmasking biases and navigating pitfalls in the ophthalmic artificial intelligence lifecycle: A narrative review

PLOS Digital Health, Journal Year: 2024, Volume and Issue: 3(10), P. e0000618 - e0000618

Published: Oct. 8, 2024

Over the past 2 decades, exponential growth in data availability, computational power, and newly available modeling techniques has led to an expansion interest, investment, research Artificial Intelligence (AI) applications. Ophthalmology is one of many fields that seek benefit from AI given advent telemedicine screening programs use ancillary imaging. However, before can be widely deployed, further work must done avoid pitfalls within lifecycle. This review article breaks down lifecycle into seven steps-data collection; defining model task; preprocessing labeling; development; evaluation validation; deployment; finally, post-deployment evaluation, monitoring, system recalibration-and delves risks for harm at each step strategies mitigating them.

Language: Английский

---

Machine learning for medical image classification

Academia Medicine, Journal Year: 2024, Volume and Issue: 1(4)

Published: Dec. 23, 2024

This review article focuses on the application of machine learning (ML) algorithms in medical image classification. It highlights intricate process involved selecting most suitable ML algorithm for predicting specific conditions, emphasizing critical role real-world data testing and validation. navigates through various methods utilized healthcare, including Supervised Learning, Unsupervised Self-Supervised Deep Neural Networks, Reinforcement Ensemble Methods. The challenge lies not just selection an but identifying appropriate one a task as well, given vast array options available. Each unique dataset requires comparative analysis to determine best-performing algorithm. However, all available is impractical. examines performance recent studies, focusing their applications across different imaging modalities diagnosing conditions. provides summary these offering starting point those seeking select conditions modalities.

Language: Английский

---

Multi-Slice Generation sMRI and fMRI for Autism Spectrum Disorder Diagnosis Using 3D-CNN and Vision Transformers

Brain Sciences, Journal Year: 2023, Volume and Issue: 13(11), P. 1578 - 1578

Published: Nov. 10, 2023

Researchers have explored various potential indicators of ASD, including changes in brain structure and activity, genetics, immune system abnormalities, but no definitive indicator has been found yet. Therefore, this study aims to investigate ASD using two types magnetic resonance images (MRI), structural (sMRI) functional (fMRI), address the issue limited data availability. Transfer learning is a valuable technique when working with data, as it utilizes knowledge gained from pre-trained model domain abundant data. This proposed use four vision transformers namely ConvNeXT, MobileNet, Swin, ViT sMRI modalities. The also investigated 3D-CNN fMRI Our experiments involved different methods generating extracting slices raw 3D 4D scans along axial, coronal, sagittal planes. To evaluate our methods, we utilized standard neuroimaging dataset called NYU ABIDE repository classify subjects typical control subjects. performance models was evaluated against several baselines studies that implemented VGG ResNet transfer models. experimental results validate effectiveness multi-slice generation they achieved state-of-the-art results. In particular, 50-middle showed profound promise classifiability obtained maximum accuracy 0.8710 F1-score 0.8261 mean across sagittal. Additionally, whole except beginnings ends views helped reduce irrelevant information good 0.8387 0.7727 F1-score. Lastly, ConvNeXt higher than other

Language: Английский

---

Integrating Retinal Segmentation Metrics with Machine Learning for Predictions from Mouse SD-OCT Scans

Current Eye Research, Journal Year: 2025, Volume and Issue: unknown, P. 1 - 10

Published: Jan. 23, 2025

Purpose This study aimed to initially test whether machine learning approaches could categorically predict two simple biological features, mouse age and species, using the retinal segmentation metrics.

Language: Английский

---

Investigating the Quality of DermaMNIST and Fitzpatrick17k Dermatological Image Datasets

Scientific Data, Journal Year: 2025, Volume and Issue: 12(1)

Published: Feb. 1, 2025

Language: Английский

---